2024-01-16T14:43:09.796 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-01-16T14:43:09.797 DEBUG:teuthology.run:Teuthology command: teuthology --description fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone-progress}} --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2024-01-16_14:37:50-fs:volumes-rishabh-clone-stats-testing-default-smithi/7518192 --verbose --name rishabh-2024-01-16_14:37:50-fs:volumes-rishabh-clone-stats-testing-default-smithi -- /home/teuthworker/archive/rishabh-2024-01-16_14:37:50-fs:volumes-rishabh-clone-stats-testing-default-smithi/7518192/orig.config.yaml 2024-01-16T14:43:09.841 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-16T14:43:09.918 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-01-16_14:37:50-fs:volumes-rishabh-clone-stats-testing-default-smithi/7518192 branch: rishabh-clone-stats description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone-progress}} email: null first_in_suite: false job_id: '7518192' kernel: client: branch: testing flavor: default kdb: true sha1: 2668949e0971849893a3fb7f177ce9032fc65780 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2024-01-16_14:37:50-fs:volumes-rishabh-clone-stats-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: rishabh-clone-stats ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 73a4532fc4d97adee0b985d4d42874ea351a1c86 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 73a4532fc4d97adee0b985d4d42874ea351a1c86 kclient: syntax: v1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: rishabh-clone-stats sha1: 73a4532fc4d97adee0b985d4d42874ea351a1c86 owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 2668 sha1: 73a4532fc4d97adee0b985d4d42874ea351a1c86 sleep_before_teardown: 0 suite: fs:volumes suite_branch: rishabh-clone-stats suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_73a4532fc4d97adee0b985d4d42874ea351a1c86/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 73a4532fc4d97adee0b985d4d42874ea351a1c86 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi143.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMEZGZXmIlZiJR1TY0oddUL9M4YS37kACpY0PZj/hAK3rR3Rgn57TOa0oCsfsM1VaszdgAJjPcjoM25mb26r/gg= smithi204.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDZizvbZvwbqCGGr+goAs4E6aM8VEU0pKCgbz9zOce6zdfM6kFS6mZWPBiywmG0ueu1N6AZFIvZ0tRGoXNgBgAY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestCloneProgress teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: cd45576300487d997e5a85abed65500b9f5d143b timestamp: 2024-01-16_14:37:50 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-01-16T14:43:09.918 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_73a4532fc4d97adee0b985d4d42874ea351a1c86/qa; will attempt to use it 2024-01-16T14:43:09.919 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_73a4532fc4d97adee0b985d4d42874ea351a1c86/qa/tasks 2024-01-16T14:43:09.919 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-01-16T14:43:09.920 INFO:teuthology.task.internal:Checking packages... 2024-01-16T14:43:09.944 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '73a4532fc4d97adee0b985d4d42874ea351a1c86' 2024-01-16T14:43:09.944 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-01-16T14:43:09.944 INFO:teuthology.packaging:ref: None 2024-01-16T14:43:09.944 INFO:teuthology.packaging:tag: None 2024-01-16T14:43:09.944 INFO:teuthology.packaging:branch: rishabh-clone-stats 2024-01-16T14:43:09.944 INFO:teuthology.packaging:sha1: 73a4532fc4d97adee0b985d4d42874ea351a1c86 2024-01-16T14:43:09.944 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=rishabh-clone-stats 2024-01-16T14:43:10.220 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-7629.g73a4532f 2024-01-16T14:43:10.221 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-01-16T14:43:10.227 INFO:teuthology.task.internal:no buildpackages task found 2024-01-16T14:43:10.228 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-01-16T14:43:10.248 INFO:teuthology.task.internal:Saving configuration 2024-01-16T14:43:10.265 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-01-16T14:43:10.271 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-16T14:43:10.302 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi143.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-01-16_14:37:50-fs:volumes-rishabh-clone-stats-testing-default-smithi/7518192', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-16 14:38:01.846038', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMEZGZXmIlZiJR1TY0oddUL9M4YS37kACpY0PZj/hAK3rR3Rgn57TOa0oCsfsM1VaszdgAJjPcjoM25mb26r/gg='} 2024-01-16T14:43:10.322 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi204.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-01-16_14:37:50-fs:volumes-rishabh-clone-stats-testing-default-smithi/7518192', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-16 14:38:01.844461', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDZizvbZvwbqCGGr+goAs4E6aM8VEU0pKCgbz9zOce6zdfM6kFS6mZWPBiywmG0ueu1N6AZFIvZ0tRGoXNgBgAY='} 2024-01-16T14:43:10.322 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-01-16T14:43:10.328 INFO:teuthology.task.internal:roles: ubuntu@smithi143.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-01-16T14:43:10.329 INFO:teuthology.task.internal:roles: ubuntu@smithi204.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-01-16T14:43:10.329 INFO:teuthology.run_tasks:Running task console_log... 2024-01-16T14:43:10.389 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4178b09160>, signals=[15]) 2024-01-16T14:43:10.390 INFO:teuthology.run_tasks:Running task internal.connect... 2024-01-16T14:43:10.396 INFO:teuthology.task.internal:Opening connections... 2024-01-16T14:43:10.396 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi143.front.sepia.ceph.com 2024-01-16T14:43:10.398 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-16T14:43:10.489 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi204.front.sepia.ceph.com 2024-01-16T14:43:10.489 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-16T14:43:10.574 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-01-16T14:43:10.581 DEBUG:teuthology.orchestra.run.smithi143:> uname -m 2024-01-16T14:43:10.598 INFO:teuthology.orchestra.run.smithi143.stdout:x86_64 2024-01-16T14:43:10.599 DEBUG:teuthology.orchestra.run.smithi143:> cat /etc/os-release 2024-01-16T14:43:10.655 INFO:teuthology.orchestra.run.smithi143.stdout:NAME="CentOS Stream" 2024-01-16T14:43:10.655 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION="9" 2024-01-16T14:43:10.655 INFO:teuthology.orchestra.run.smithi143.stdout:ID="centos" 2024-01-16T14:43:10.655 INFO:teuthology.orchestra.run.smithi143.stdout:ID_LIKE="rhel fedora" 2024-01-16T14:43:10.656 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION_ID="9" 2024-01-16T14:43:10.656 INFO:teuthology.orchestra.run.smithi143.stdout:PLATFORM_ID="platform:el9" 2024-01-16T14:43:10.656 INFO:teuthology.orchestra.run.smithi143.stdout:PRETTY_NAME="CentOS Stream 9" 2024-01-16T14:43:10.656 INFO:teuthology.orchestra.run.smithi143.stdout:ANSI_COLOR="0;31" 2024-01-16T14:43:10.656 INFO:teuthology.orchestra.run.smithi143.stdout:LOGO="fedora-logo-icon" 2024-01-16T14:43:10.656 INFO:teuthology.orchestra.run.smithi143.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-01-16T14:43:10.656 INFO:teuthology.orchestra.run.smithi143.stdout:HOME_URL="https://centos.org/" 2024-01-16T14:43:10.656 INFO:teuthology.orchestra.run.smithi143.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-16T14:43:10.656 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-01-16T14:43:10.656 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-01-16T14:43:10.657 INFO:teuthology.lock.ops:Updating smithi143.front.sepia.ceph.com on lock server 2024-01-16T14:43:10.676 DEBUG:teuthology.orchestra.run.smithi204:> uname -m 2024-01-16T14:43:10.694 INFO:teuthology.orchestra.run.smithi204.stdout:x86_64 2024-01-16T14:43:10.694 DEBUG:teuthology.orchestra.run.smithi204:> cat /etc/os-release 2024-01-16T14:43:10.750 INFO:teuthology.orchestra.run.smithi204.stdout:NAME="CentOS Stream" 2024-01-16T14:43:10.750 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION="9" 2024-01-16T14:43:10.750 INFO:teuthology.orchestra.run.smithi204.stdout:ID="centos" 2024-01-16T14:43:10.750 INFO:teuthology.orchestra.run.smithi204.stdout:ID_LIKE="rhel fedora" 2024-01-16T14:43:10.751 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION_ID="9" 2024-01-16T14:43:10.751 INFO:teuthology.orchestra.run.smithi204.stdout:PLATFORM_ID="platform:el9" 2024-01-16T14:43:10.751 INFO:teuthology.orchestra.run.smithi204.stdout:PRETTY_NAME="CentOS Stream 9" 2024-01-16T14:43:10.751 INFO:teuthology.orchestra.run.smithi204.stdout:ANSI_COLOR="0;31" 2024-01-16T14:43:10.751 INFO:teuthology.orchestra.run.smithi204.stdout:LOGO="fedora-logo-icon" 2024-01-16T14:43:10.751 INFO:teuthology.orchestra.run.smithi204.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-01-16T14:43:10.751 INFO:teuthology.orchestra.run.smithi204.stdout:HOME_URL="https://centos.org/" 2024-01-16T14:43:10.751 INFO:teuthology.orchestra.run.smithi204.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-16T14:43:10.751 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-01-16T14:43:10.751 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-01-16T14:43:10.751 INFO:teuthology.lock.ops:Updating smithi204.front.sepia.ceph.com on lock server 2024-01-16T14:43:10.776 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-01-16T14:43:10.786 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-01-16T14:43:10.791 INFO:teuthology.task.internal:Checking for old test directory... 2024-01-16T14:43:10.792 DEBUG:teuthology.orchestra.run.smithi143:> test '!' -e /home/ubuntu/cephtest 2024-01-16T14:43:10.794 DEBUG:teuthology.orchestra.run.smithi204:> test '!' -e /home/ubuntu/cephtest 2024-01-16T14:43:10.809 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-01-16T14:43:10.815 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-01-16T14:43:10.815 DEBUG:teuthology.orchestra.run.smithi143:> test -z $(ls -A /var/lib/ceph) 2024-01-16T14:43:10.851 DEBUG:teuthology.orchestra.run.smithi204:> test -z $(ls -A /var/lib/ceph) 2024-01-16T14:43:10.962 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-01-16T14:43:11.018 INFO:teuthology.run_tasks:Running task kernel... 2024-01-16T14:43:11.033 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '2668949e0971849893a3fb7f177ce9032fc65780'} 2024-01-16T14:43:11.033 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-01-16T14:43:11.033 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-01-16T14:43:11.034 DEBUG:teuthology.orchestra.run.smithi143:> test -f /run/.containerenv -o -f /.dockerenv 2024-01-16T14:43:11.034 DEBUG:teuthology.orchestra.run.smithi204:> test -f /run/.containerenv -o -f /.dockerenv 2024-01-16T14:43:11.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-16T14:43:11.051 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-01-16T14:43:11.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-16T14:43:11.054 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-01-16T14:43:11.221 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2668949e0971849893a3fb7f177ce9032fc65780 2024-01-16T14:43:11.221 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2668949e0971849893a3fb7f177ce9032fc65780"... 2024-01-16T14:43:11.221 DEBUG:teuthology.orchestra.run.smithi143:> uname -r 2024-01-16T14:43:11.235 DEBUG:teuthology.task.kernel:sha1 for client.1 is 2668949e0971849893a3fb7f177ce9032fc65780 2024-01-16T14:43:11.235 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "2668949e0971849893a3fb7f177ce9032fc65780"... 2024-01-16T14:43:11.236 DEBUG:teuthology.orchestra.run.smithi204:> uname -r 2024-01-16T14:43:11.238 INFO:teuthology.orchestra.run.smithi143.stdout:5.14.0-361.el9.x86_64 2024-01-16T14:43:11.238 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 2668949e0971849893a3fb7f177ce9032fc65780 2024-01-16T14:43:11.239 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-01-16T14:43:11.239 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum upgrade -y linux-firmware 2024-01-16T14:43:11.253 INFO:teuthology.orchestra.run.smithi204.stdout:5.14.0-361.el9.x86_64 2024-01-16T14:43:11.253 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 2668949e0971849893a3fb7f177ce9032fc65780 2024-01-16T14:43:11.253 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-01-16T14:43:11.253 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum upgrade -y linux-firmware 2024-01-16T14:43:13.997 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS-9 - AppStream 5.5 MB/s | 6.4 MB 00:01 2024-01-16T14:43:14.020 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS-9 - AppStream 6.5 MB/s | 6.4 MB 00:00 2024-01-16T14:43:15.770 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS-9 - BaseOS 2.8 MB/s | 2.0 MB 00:00 2024-01-16T14:43:15.963 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS-9 - BaseOS 2.2 MB/s | 2.0 MB 00:00 2024-01-16T14:43:17.952 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.0 MB 00:01 2024-01-16T14:43:18.046 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - BaseOS 4.5 MB/s | 8.0 MB 00:01 2024-01-16T14:43:23.778 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - AppStream 3.9 MB/s | 19 MB 00:04 2024-01-16T14:43:25.112 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - AppStream 3.1 MB/s | 19 MB 00:05 2024-01-16T14:43:29.231 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - CRB 2.9 MB/s | 5.9 MB 00:02 2024-01-16T14:43:30.762 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 15 kB 00:00 2024-01-16T14:43:30.891 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - CRB 2.6 MB/s | 5.9 MB 00:02 2024-01-16T14:43:31.605 INFO:teuthology.orchestra.run.smithi204.stdout:Extra Packages for Enterprise Linux 28 MB/s | 20 MB 00:00 2024-01-16T14:43:32.564 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 15 kB 00:00 2024-01-16T14:43:34.876 INFO:teuthology.orchestra.run.smithi143.stdout:Extra Packages for Enterprise Linux 9.2 MB/s | 20 MB 00:02 2024-01-16T14:43:36.330 INFO:teuthology.orchestra.run.smithi204.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-01-16T14:43:38.609 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-01-16T14:43:38.619 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-01-16T14:43:38.619 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-01-16T14:43:38.619 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-01-16T14:43:38.619 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrading: 2024-01-16T14:43:38.620 INFO:teuthology.orchestra.run.smithi204.stdout: linux-firmware noarch 20240109-142.el9 CentOS-BaseOS 375 M 2024-01-16T14:43:38.620 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:43:38.620 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-01-16T14:43:38.620 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-01-16T14:43:38.620 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrade 1 Package 2024-01-16T14:43:38.620 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:43:38.620 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 375 M 2024-01-16T14:43:38.620 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-01-16T14:43:39.706 INFO:teuthology.orchestra.run.smithi143.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-01-16T14:43:42.055 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-01-16T14:43:42.065 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-01-16T14:43:42.065 INFO:teuthology.orchestra.run.smithi143.stdout: Package Arch Version Repository Size 2024-01-16T14:43:42.065 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-01-16T14:43:42.066 INFO:teuthology.orchestra.run.smithi143.stdout:Upgrading: 2024-01-16T14:43:42.066 INFO:teuthology.orchestra.run.smithi143.stdout: linux-firmware noarch 20240109-142.el9 CentOS-BaseOS 375 M 2024-01-16T14:43:42.066 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:43:42.066 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2024-01-16T14:43:42.066 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-01-16T14:43:42.066 INFO:teuthology.orchestra.run.smithi143.stdout:Upgrade 1 Package 2024-01-16T14:43:42.066 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:43:42.066 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 375 M 2024-01-16T14:43:42.066 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2024-01-16T14:44:06.338 INFO:teuthology.orchestra.run.smithi143.stdout:linux-firmware-20240109-142.el9.noarch.rpm 15 MB/s | 375 MB 00:24 2024-01-16T14:44:06.339 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2024-01-16T14:44:06.339 INFO:teuthology.orchestra.run.smithi143.stdout:Total 15 MB/s | 375 MB 00:24 2024-01-16T14:44:06.340 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2024-01-16T14:44:06.448 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2024-01-16T14:44:06.449 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2024-01-16T14:44:08.806 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2024-01-16T14:44:08.806 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2024-01-16T14:44:15.136 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2024-01-16T14:44:15.363 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : linux-firmware-20240109-142.el9.noarch 1/2 2024-01-16T14:44:15.454 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-01-16T14:44:16.380 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-01-16T14:44:16.380 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : linux-firmware-20240109-142.el9.noarch 1/2 2024-01-16T14:44:16.783 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-01-16T14:44:16.783 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:44:16.783 INFO:teuthology.orchestra.run.smithi143.stdout:Upgraded: 2024-01-16T14:44:16.783 INFO:teuthology.orchestra.run.smithi143.stdout: linux-firmware-20240109-142.el9.noarch 2024-01-16T14:44:16.784 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:44:16.784 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-01-16T14:44:17.089 INFO:teuthology.task.kernel:Downloading kernel 2668949e0971849893a3fb7f177ce9032fc65780 on client.0... 2024-01-16T14:44:17.089 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2668949e0971849893a3fb7f177ce9032fc65780 2024-01-16T14:44:17.382 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2668949e0971849893a3fb7f177ce9032fc65780/centos/9/flavors/default/x86_64/ 2024-01-16T14:44:17.382 DEBUG:teuthology.orchestra.run.smithi143:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.7.0_g2668949e0971-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2668949e0971849893a3fb7f177ce9032fc65780/centos/9/flavors/default/x86_64/ --input-file=- 2024-01-16T14:44:17.385 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi143.front.sepia.ceph.com to complete... 2024-01-16T14:44:41.841 INFO:teuthology.orchestra.run.smithi204.stdout:linux-firmware-20240109-142.el9.noarch.rpm 5.9 MB/s | 375 MB 01:03 2024-01-16T14:44:41.841 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-01-16T14:44:41.842 INFO:teuthology.orchestra.run.smithi204.stdout:Total 5.9 MB/s | 375 MB 01:03 2024-01-16T14:44:41.842 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-01-16T14:44:41.924 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-01-16T14:44:41.925 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-01-16T14:44:44.291 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-01-16T14:44:44.292 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-01-16T14:44:50.650 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-01-16T14:44:50.907 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : linux-firmware-20240109-142.el9.noarch 1/2 2024-01-16T14:44:50.998 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-01-16T14:44:51.975 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-01-16T14:44:51.975 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : linux-firmware-20240109-142.el9.noarch 1/2 2024-01-16T14:44:52.404 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-01-16T14:44:52.404 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:44:52.405 INFO:teuthology.orchestra.run.smithi204.stdout:Upgraded: 2024-01-16T14:44:52.405 INFO:teuthology.orchestra.run.smithi204.stdout: linux-firmware-20240109-142.el9.noarch 2024-01-16T14:44:52.405 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:44:52.405 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-01-16T14:44:52.692 INFO:teuthology.task.kernel:Downloading kernel 2668949e0971849893a3fb7f177ce9032fc65780 on client.1... 2024-01-16T14:44:52.693 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2668949e0971849893a3fb7f177ce9032fc65780 2024-01-16T14:44:53.074 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2668949e0971849893a3fb7f177ce9032fc65780/centos/9/flavors/default/x86_64/ 2024-01-16T14:44:53.075 DEBUG:teuthology.orchestra.run.smithi204:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.7.0_g2668949e0971-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2668949e0971849893a3fb7f177ce9032fc65780/centos/9/flavors/default/x86_64/ --input-file=- 2024-01-16T14:44:53.077 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi204.front.sepia.ceph.com to complete... 2024-01-16T14:47:45.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16 14:47:45 URL:https://3.chacra.ceph.com/r/kernel/testing/2668949e0971849893a3fb7f177ce9032fc65780/centos/9/flavors/default/x86_64/kernel-6.7.0_g2668949e0971-1.x86_64.rpm [330271041/330271041] -> "/tmp/kernel.x86_64.rpm" [1] 2024-01-16T14:47:45.767 INFO:teuthology.orchestra.run.smithi143.stderr:FINISHED --2024-01-16 14:47:45-- 2024-01-16T14:47:45.767 INFO:teuthology.orchestra.run.smithi143.stderr:Total wall clock time: 3m 28s 2024-01-16T14:47:45.767 INFO:teuthology.orchestra.run.smithi143.stderr:Downloaded: 1 files, 315M in 3m 28s (1.51 MB/s) 2024-01-16T14:47:45.771 INFO:teuthology.task.kernel:Installing kernel 2668949e0971849893a3fb7f177ce9032fc65780 on client.0... 2024-01-16T14:47:45.771 DEBUG:teuthology.orchestra.run.smithi143:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-01-16T14:47:47.675 INFO:teuthology.orchestra.run.smithi143.stdout:Verifying... ######################################## 2024-01-16T14:47:47.819 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing... ######################################## 2024-01-16T14:47:47.898 INFO:teuthology.orchestra.run.smithi143.stdout:Updating / installing... 2024-01-16T14:47:55.280 INFO:teuthology.orchestra.run.smithi143.stdout:kernel-6.7.0_g2668949e0971-1 ######################################## 2024-01-16T14:48:03.985 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16 14:48:03 URL:https://3.chacra.ceph.com/r/kernel/testing/2668949e0971849893a3fb7f177ce9032fc65780/centos/9/flavors/default/x86_64/kernel-6.7.0_g2668949e0971-1.x86_64.rpm [330271041/330271041] -> "/tmp/kernel.x86_64.rpm" [1] 2024-01-16T14:48:03.986 INFO:teuthology.orchestra.run.smithi204.stderr:FINISHED --2024-01-16 14:48:03-- 2024-01-16T14:48:03.986 INFO:teuthology.orchestra.run.smithi204.stderr:Total wall clock time: 3m 11s 2024-01-16T14:48:03.986 INFO:teuthology.orchestra.run.smithi204.stderr:Downloaded: 1 files, 315M in 3m 10s (1.66 MB/s) 2024-01-16T14:48:03.989 INFO:teuthology.task.kernel:Installing kernel 2668949e0971849893a3fb7f177ce9032fc65780 on client.1... 2024-01-16T14:48:03.989 DEBUG:teuthology.orchestra.run.smithi204:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-01-16T14:48:05.885 INFO:teuthology.orchestra.run.smithi204.stdout:Verifying... ######################################## 2024-01-16T14:48:06.028 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing... ######################################## 2024-01-16T14:48:06.107 INFO:teuthology.orchestra.run.smithi204.stdout:Updating / installing... 2024-01-16T14:48:10.099 INFO:teuthology.orchestra.run.smithi143.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-01-16T14:48:13.495 INFO:teuthology.orchestra.run.smithi204.stdout:kernel-6.7.0_g2668949e0971-1 ######################################## 2024-01-16T14:48:28.174 INFO:teuthology.orchestra.run.smithi204.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-01-16T14:48:32.272 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi143.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-01-16T14:48:32.273 DEBUG:teuthology.orchestra.run.smithi143:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-01-16T14:48:32.355 INFO:teuthology.orchestra.run.smithi143.stdout:/boot/System.map-6.7.0-g2668949e0971 2024-01-16T14:48:32.355 INFO:teuthology.orchestra.run.smithi143.stdout:/boot/config-6.7.0-g2668949e0971 2024-01-16T14:48:32.355 INFO:teuthology.orchestra.run.smithi143.stdout:/boot/vmlinuz-6.7.0-g2668949e0971 2024-01-16T14:48:32.355 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971 2024-01-16T14:48:32.355 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel 2024-01-16T14:48:32.355 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch 2024-01-16T14:48:32.355 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86 2024-01-16T14:48:32.355 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto 2024-01-16T14:48:32.356 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-01-16T14:48:32.356 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-01-16T14:48:32.356 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-01-16T14:48:32.356 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-01-16T14:48:32.356 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-01-16T14:48:32.356 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-01-16T14:48:32.356 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-01-16T14:48:32.356 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/crc32c-intel.ko 2024-01-16T14:48:32.356 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-01-16T14:48:32.356 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-01-16T14:48:32.356 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-01-16T14:48:32.356 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-01-16T14:48:32.356 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/serpent-avx2.ko 2024-01-16T14:48:32.357 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-01-16T14:48:32.357 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-01-16T14:48:32.357 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-01-16T14:48:32.357 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-01-16T14:48:32.357 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-01-16T14:48:32.357 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-01-16T14:48:32.357 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-01-16T14:48:32.357 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kernel 2024-01-16T14:48:32.357 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kernel/cpu 2024-01-16T14:48:32.357 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kernel/cpu/mce 2024-01-16T14:48:32.357 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-01-16T14:48:32.357 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kvm 2024-01-16T14:48:32.357 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kvm/kvm-amd.ko 2024-01-16T14:48:32.357 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kvm/kvm-intel.ko 2024-01-16T14:48:32.357 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kvm/kvm.ko 2024-01-16T14:48:32.358 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto 2024-01-16T14:48:32.358 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/ansi_cprng.ko 2024-01-16T14:48:32.358 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/anubis.ko 2024-01-16T14:48:32.358 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/arc4.ko 2024-01-16T14:48:32.358 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx 2024-01-16T14:48:32.358 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx/async_memcpy.ko 2024-01-16T14:48:32.358 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx/async_pq.ko 2024-01-16T14:48:32.358 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx/async_raid6_recov.ko 2024-01-16T14:48:32.358 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx/async_tx.ko 2024-01-16T14:48:32.358 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx/async_xor.ko 2024-01-16T14:48:32.358 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx/raid6test.ko 2024-01-16T14:48:32.358 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/authenc.ko 2024-01-16T14:48:32.358 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/authencesn.ko 2024-01-16T14:48:32.358 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/blake2b_generic.ko 2024-01-16T14:48:32.358 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/blowfish_common.ko 2024-01-16T14:48:32.359 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/blowfish_generic.ko 2024-01-16T14:48:32.359 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/camellia_generic.ko 2024-01-16T14:48:32.359 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/cast5_generic.ko 2024-01-16T14:48:32.359 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/cast6_generic.ko 2024-01-16T14:48:32.359 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/cast_common.ko 2024-01-16T14:48:32.359 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/ccm.ko 2024-01-16T14:48:32.359 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/cmac.ko 2024-01-16T14:48:32.359 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/crc32_generic.ko 2024-01-16T14:48:32.359 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/crypto_engine.ko 2024-01-16T14:48:32.359 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/crypto_user.ko 2024-01-16T14:48:32.359 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/cts.ko 2024-01-16T14:48:32.359 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/des_generic.ko 2024-01-16T14:48:32.359 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/dh_generic.ko 2024-01-16T14:48:32.359 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/echainiv.ko 2024-01-16T14:48:32.359 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/essiv.ko 2024-01-16T14:48:32.360 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/fcrypt.ko 2024-01-16T14:48:32.360 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/khazad.ko 2024-01-16T14:48:32.360 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/lz4.ko 2024-01-16T14:48:32.360 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/lz4hc.ko 2024-01-16T14:48:32.360 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/md4.ko 2024-01-16T14:48:32.360 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/michael_mic.ko 2024-01-16T14:48:32.360 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/pcbc.ko 2024-01-16T14:48:32.360 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/pcrypt.ko 2024-01-16T14:48:32.360 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/rmd160.ko 2024-01-16T14:48:32.360 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/seed.ko 2024-01-16T14:48:32.360 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/serpent_generic.ko 2024-01-16T14:48:32.360 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/tcrypt.ko 2024-01-16T14:48:32.360 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/tea.ko 2024-01-16T14:48:32.360 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/twofish_common.ko 2024-01-16T14:48:32.361 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/twofish_generic.ko 2024-01-16T14:48:32.361 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/vmac.ko 2024-01-16T14:48:32.361 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/wp512.ko 2024-01-16T14:48:32.361 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/xcbc.ko 2024-01-16T14:48:32.361 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/xor.ko 2024-01-16T14:48:32.361 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/xxhash_generic.ko 2024-01-16T14:48:32.361 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers 2024-01-16T14:48:32.361 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi 2024-01-16T14:48:32.361 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/acpi_ipmi.ko 2024-01-16T14:48:32.361 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/acpi_pad.ko 2024-01-16T14:48:32.361 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/custom_method.ko 2024-01-16T14:48:32.361 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/ec_sys.ko 2024-01-16T14:48:32.361 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/platform_profile.ko 2024-01-16T14:48:32.361 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/sbs.ko 2024-01-16T14:48:32.362 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/sbshc.ko 2024-01-16T14:48:32.362 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/video.ko 2024-01-16T14:48:32.362 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata 2024-01-16T14:48:32.362 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/acard-ahci.ko 2024-01-16T14:48:32.362 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/ahci_platform.ko 2024-01-16T14:48:32.362 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/ata_generic.ko 2024-01-16T14:48:32.362 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/libahci_platform.ko 2024-01-16T14:48:32.362 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_acpi.ko 2024-01-16T14:48:32.362 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_ali.ko 2024-01-16T14:48:32.362 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_amd.ko 2024-01-16T14:48:32.362 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_artop.ko 2024-01-16T14:48:32.362 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_atiixp.ko 2024-01-16T14:48:32.362 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_atp867x.ko 2024-01-16T14:48:32.362 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_cmd640.ko 2024-01-16T14:48:32.362 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_cmd64x.ko 2024-01-16T14:48:32.362 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_cypress.ko 2024-01-16T14:48:32.363 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_efar.ko 2024-01-16T14:48:32.363 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_hpt366.ko 2024-01-16T14:48:32.363 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_hpt37x.ko 2024-01-16T14:48:32.363 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_hpt3x2n.ko 2024-01-16T14:48:32.363 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_hpt3x3.ko 2024-01-16T14:48:32.363 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_it8213.ko 2024-01-16T14:48:32.363 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_it821x.ko 2024-01-16T14:48:32.363 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_jmicron.ko 2024-01-16T14:48:32.363 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_marvell.ko 2024-01-16T14:48:32.363 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_mpiix.ko 2024-01-16T14:48:32.363 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_netcell.ko 2024-01-16T14:48:32.363 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_ninja32.ko 2024-01-16T14:48:32.363 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_ns87410.ko 2024-01-16T14:48:32.363 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_ns87415.ko 2024-01-16T14:48:32.364 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_oldpiix.ko 2024-01-16T14:48:32.364 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_opti.ko 2024-01-16T14:48:32.364 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_optidma.ko 2024-01-16T14:48:32.364 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_pcmcia.ko 2024-01-16T14:48:32.364 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_pdc2027x.ko 2024-01-16T14:48:32.364 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-01-16T14:48:32.364 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_piccolo.ko 2024-01-16T14:48:32.364 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_rdc.ko 2024-01-16T14:48:32.364 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_sch.ko 2024-01-16T14:48:32.364 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_serverworks.ko 2024-01-16T14:48:32.364 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_sil680.ko 2024-01-16T14:48:32.364 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_sis.ko 2024-01-16T14:48:32.364 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_sl82c105.ko 2024-01-16T14:48:32.364 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_triflex.ko 2024-01-16T14:48:32.365 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_via.ko 2024-01-16T14:48:32.365 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pdc_adma.ko 2024-01-16T14:48:32.365 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_inic162x.ko 2024-01-16T14:48:32.365 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_mv.ko 2024-01-16T14:48:32.365 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_nv.ko 2024-01-16T14:48:32.365 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_promise.ko 2024-01-16T14:48:32.365 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_qstor.ko 2024-01-16T14:48:32.365 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_sil.ko 2024-01-16T14:48:32.365 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_sil24.ko 2024-01-16T14:48:32.365 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_sis.ko 2024-01-16T14:48:32.365 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_svw.ko 2024-01-16T14:48:32.365 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_sx4.ko 2024-01-16T14:48:32.365 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_uli.ko 2024-01-16T14:48:32.365 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_via.ko 2024-01-16T14:48:32.365 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_vsc.ko 2024-01-16T14:48:32.366 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/auxdisplay 2024-01-16T14:48:32.366 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/auxdisplay/ks0108.ko 2024-01-16T14:48:32.366 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/base 2024-01-16T14:48:32.366 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/base/regmap 2024-01-16T14:48:32.366 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/base/regmap/regmap-i2c.ko 2024-01-16T14:48:32.366 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/base/regmap/regmap-mmio.ko 2024-01-16T14:48:32.366 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/bcma 2024-01-16T14:48:32.366 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/bcma/bcma.ko 2024-01-16T14:48:32.366 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block 2024-01-16T14:48:32.366 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/aoe 2024-01-16T14:48:32.366 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/aoe/aoe.ko 2024-01-16T14:48:32.366 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/brd.ko 2024-01-16T14:48:32.366 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/drbd 2024-01-16T14:48:32.366 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/drbd/drbd.ko 2024-01-16T14:48:32.367 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/floppy.ko 2024-01-16T14:48:32.367 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/loop.ko 2024-01-16T14:48:32.367 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/mtip32xx 2024-01-16T14:48:32.367 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-01-16T14:48:32.367 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/nbd.ko 2024-01-16T14:48:32.367 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/null_blk 2024-01-16T14:48:32.367 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/null_blk/null_blk.ko 2024-01-16T14:48:32.367 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/pktcdvd.ko 2024-01-16T14:48:32.367 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/rbd.ko 2024-01-16T14:48:32.367 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/virtio_blk.ko 2024-01-16T14:48:32.367 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/xen-blkback 2024-01-16T14:48:32.367 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-01-16T14:48:32.367 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/xen-blkfront.ko 2024-01-16T14:48:32.367 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/zram 2024-01-16T14:48:32.367 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/zram/zram.ko 2024-01-16T14:48:32.367 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char 2024-01-16T14:48:32.368 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hangcheck-timer.ko 2024-01-16T14:48:32.368 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hw_random 2024-01-16T14:48:32.368 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hw_random/amd-rng.ko 2024-01-16T14:48:32.368 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hw_random/intel-rng.ko 2024-01-16T14:48:32.368 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-01-16T14:48:32.368 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hw_random/via-rng.ko 2024-01-16T14:48:32.368 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hw_random/virtio-rng.ko 2024-01-16T14:48:32.368 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi 2024-01-16T14:48:32.368 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-01-16T14:48:32.368 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-01-16T14:48:32.368 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-01-16T14:48:32.368 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi/ipmi_si.ko 2024-01-16T14:48:32.368 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-01-16T14:48:32.368 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-01-16T14:48:32.368 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/lp.ko 2024-01-16T14:48:32.369 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/mwave 2024-01-16T14:48:32.369 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/mwave/mwave.ko 2024-01-16T14:48:32.369 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ppdev.ko 2024-01-16T14:48:32.369 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tlclk.ko 2024-01-16T14:48:32.369 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm 2024-01-16T14:48:32.369 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/tpm.ko 2024-01-16T14:48:32.369 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/tpm_atmel.ko 2024-01-16T14:48:32.369 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/tpm_infineon.ko 2024-01-16T14:48:32.369 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/tpm_nsc.ko 2024-01-16T14:48:32.369 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/tpm_tis.ko 2024-01-16T14:48:32.369 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-01-16T14:48:32.369 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/uv_mmtimer.ko 2024-01-16T14:48:32.369 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/virtio_console.ko 2024-01-16T14:48:32.369 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/cpufreq 2024-01-16T14:48:32.370 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-01-16T14:48:32.370 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-01-16T14:48:32.370 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/cpufreq/p4-clockmod.ko 2024-01-16T14:48:32.370 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-01-16T14:48:32.370 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/cpufreq/powernow-k8.ko 2024-01-16T14:48:32.370 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/cpufreq/speedstep-lib.ko 2024-01-16T14:48:32.370 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto 2024-01-16T14:48:32.370 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/ccp 2024-01-16T14:48:32.370 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-01-16T14:48:32.370 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/ccp/ccp.ko 2024-01-16T14:48:32.370 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/intel 2024-01-16T14:48:32.370 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/intel/qat 2024-01-16T14:48:32.370 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/intel/qat/qat_common 2024-01-16T14:48:32.370 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-01-16T14:48:32.371 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-01-16T14:48:32.371 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-01-16T14:48:32.371 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/padlock-aes.ko 2024-01-16T14:48:32.371 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/padlock-sha.ko 2024-01-16T14:48:32.371 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/virtio 2024-01-16T14:48:32.371 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-01-16T14:48:32.371 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/dca 2024-01-16T14:48:32.371 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/dca/dca.ko 2024-01-16T14:48:32.371 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/devfreq 2024-01-16T14:48:32.371 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-01-16T14:48:32.371 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/dma 2024-01-16T14:48:32.371 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/dma/dw 2024-01-16T14:48:32.372 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/dma/dw/dw_dmac.ko 2024-01-16T14:48:32.372 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/dma/ioat 2024-01-16T14:48:32.372 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/dma/ioat/ioatdma.ko 2024-01-16T14:48:32.372 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac 2024-01-16T14:48:32.372 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/amd64_edac.ko 2024-01-16T14:48:32.372 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/e752x_edac.ko 2024-01-16T14:48:32.372 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/edac_mce_amd.ko 2024-01-16T14:48:32.372 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i3000_edac.ko 2024-01-16T14:48:32.372 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i3200_edac.ko 2024-01-16T14:48:32.372 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i5100_edac.ko 2024-01-16T14:48:32.372 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i5400_edac.ko 2024-01-16T14:48:32.372 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i7300_edac.ko 2024-01-16T14:48:32.372 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i7core_edac.ko 2024-01-16T14:48:32.372 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i82975x_edac.ko 2024-01-16T14:48:32.373 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/ie31200_edac.ko 2024-01-16T14:48:32.373 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/sb_edac.ko 2024-01-16T14:48:32.373 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/x38_edac.ko 2024-01-16T14:48:32.373 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/firmware 2024-01-16T14:48:32.373 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/firmware/edd.ko 2024-01-16T14:48:32.373 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/firmware/iscsi_ibft.ko 2024-01-16T14:48:32.373 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio 2024-01-16T14:48:32.373 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-ich.ko 2024-01-16T14:48:32.373 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-viperboard.ko 2024-01-16T14:48:32.373 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid 2024-01-16T14:48:32.373 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-appleir.ko 2024-01-16T14:48:32.373 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-aureal.ko 2024-01-16T14:48:32.373 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-axff.ko 2024-01-16T14:48:32.373 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-dr.ko 2024-01-16T14:48:32.373 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-elecom.ko 2024-01-16T14:48:32.374 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-elo.ko 2024-01-16T14:48:32.374 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-emsff.ko 2024-01-16T14:48:32.374 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-gaff.ko 2024-01-16T14:48:32.374 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-gt683r.ko 2024-01-16T14:48:32.374 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-gyration.ko 2024-01-16T14:48:32.374 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-holtek-kbd.ko 2024-01-16T14:48:32.374 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-holtek-mouse.ko 2024-01-16T14:48:32.374 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-holtekff.ko 2024-01-16T14:48:32.374 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-hyperv.ko 2024-01-16T14:48:32.374 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-icade.ko 2024-01-16T14:48:32.374 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-keytouch.ko 2024-01-16T14:48:32.374 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-kye.ko 2024-01-16T14:48:32.374 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-lcpower.ko 2024-01-16T14:48:32.375 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-led.ko 2024-01-16T14:48:32.375 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-lenovo.ko 2024-01-16T14:48:32.375 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-logitech-dj.ko 2024-01-16T14:48:32.375 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-01-16T14:48:32.375 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-multitouch.ko 2024-01-16T14:48:32.375 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-ortek.ko 2024-01-16T14:48:32.375 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-penmount.ko 2024-01-16T14:48:32.375 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-petalynx.ko 2024-01-16T14:48:32.375 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-picolcd.ko 2024-01-16T14:48:32.375 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-pl.ko 2024-01-16T14:48:32.375 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-plantronics.ko 2024-01-16T14:48:32.375 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-primax.ko 2024-01-16T14:48:32.375 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-rmi.ko 2024-01-16T14:48:32.375 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-arvo.ko 2024-01-16T14:48:32.375 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-common.ko 2024-01-16T14:48:32.376 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-isku.ko 2024-01-16T14:48:32.376 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-kone.ko 2024-01-16T14:48:32.376 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-01-16T14:48:32.376 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-konepure.ko 2024-01-16T14:48:32.376 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-01-16T14:48:32.376 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-lua.ko 2024-01-16T14:48:32.376 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-pyra.ko 2024-01-16T14:48:32.376 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-ryos.ko 2024-01-16T14:48:32.376 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-savu.ko 2024-01-16T14:48:32.376 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat.ko 2024-01-16T14:48:32.376 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-saitek.ko 2024-01-16T14:48:32.376 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-samsung.ko 2024-01-16T14:48:32.376 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-sensor-hub.ko 2024-01-16T14:48:32.376 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-sjoy.ko 2024-01-16T14:48:32.377 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-sony.ko 2024-01-16T14:48:32.377 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-speedlink.ko 2024-01-16T14:48:32.377 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-steelseries.ko 2024-01-16T14:48:32.377 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-sunplus.ko 2024-01-16T14:48:32.377 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-thrustmaster.ko 2024-01-16T14:48:32.377 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-tivo.ko 2024-01-16T14:48:32.377 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-tmff.ko 2024-01-16T14:48:32.377 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-topseed.ko 2024-01-16T14:48:32.377 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-twinhan.ko 2024-01-16T14:48:32.377 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-uclogic.ko 2024-01-16T14:48:32.377 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-waltop.ko 2024-01-16T14:48:32.377 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-wiimote.ko 2024-01-16T14:48:32.377 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-xinmo.ko 2024-01-16T14:48:32.377 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-zpff.ko 2024-01-16T14:48:32.377 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-zydacron.ko 2024-01-16T14:48:32.378 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/uhid.ko 2024-01-16T14:48:32.378 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/wacom.ko 2024-01-16T14:48:32.378 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hv 2024-01-16T14:48:32.378 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hv/hv_balloon.ko 2024-01-16T14:48:32.378 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hv/hv_utils.ko 2024-01-16T14:48:32.378 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hv/hv_vmbus.ko 2024-01-16T14:48:32.378 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon 2024-01-16T14:48:32.378 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/abituguru.ko 2024-01-16T14:48:32.378 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/abituguru3.ko 2024-01-16T14:48:32.378 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/acpi_power_meter.ko 2024-01-16T14:48:32.378 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ad7414.ko 2024-01-16T14:48:32.378 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ad7418.ko 2024-01-16T14:48:32.378 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adc128d818.ko 2024-01-16T14:48:32.378 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adm1025.ko 2024-01-16T14:48:32.378 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adm1026.ko 2024-01-16T14:48:32.379 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adm1029.ko 2024-01-16T14:48:32.379 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adm1031.ko 2024-01-16T14:48:32.379 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adm9240.ko 2024-01-16T14:48:32.379 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ads7828.ko 2024-01-16T14:48:32.379 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adt7410.ko 2024-01-16T14:48:32.379 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adt7411.ko 2024-01-16T14:48:32.379 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adt7462.ko 2024-01-16T14:48:32.379 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adt7470.ko 2024-01-16T14:48:32.379 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adt7475.ko 2024-01-16T14:48:32.379 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adt7x10.ko 2024-01-16T14:48:32.379 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/amc6821.ko 2024-01-16T14:48:32.379 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/applesmc.ko 2024-01-16T14:48:32.379 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/asb100.ko 2024-01-16T14:48:32.379 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/asc7621.ko 2024-01-16T14:48:32.380 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/asus_atk0110.ko 2024-01-16T14:48:32.380 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/atxp1.ko 2024-01-16T14:48:32.380 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/coretemp.ko 2024-01-16T14:48:32.380 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-01-16T14:48:32.380 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/dme1737.ko 2024-01-16T14:48:32.380 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ds1621.ko 2024-01-16T14:48:32.380 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ds620.ko 2024-01-16T14:48:32.380 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/emc1403.ko 2024-01-16T14:48:32.380 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/emc6w201.ko 2024-01-16T14:48:32.380 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/f71805f.ko 2024-01-16T14:48:32.380 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/f71882fg.ko 2024-01-16T14:48:32.380 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/f75375s.ko 2024-01-16T14:48:32.380 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/fam15h_power.ko 2024-01-16T14:48:32.380 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/fschmd.ko 2024-01-16T14:48:32.381 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/g760a.ko 2024-01-16T14:48:32.381 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/g762.ko 2024-01-16T14:48:32.381 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/gl518sm.ko 2024-01-16T14:48:32.381 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/gl520sm.ko 2024-01-16T14:48:32.381 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/hwmon-vid.ko 2024-01-16T14:48:32.381 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/i5500_temp.ko 2024-01-16T14:48:32.381 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/i5k_amb.ko 2024-01-16T14:48:32.381 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ibmaem.ko 2024-01-16T14:48:32.381 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ibmpex.ko 2024-01-16T14:48:32.381 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ina209.ko 2024-01-16T14:48:32.381 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ina2xx.ko 2024-01-16T14:48:32.381 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/it87.ko 2024-01-16T14:48:32.382 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/k10temp.ko 2024-01-16T14:48:32.382 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/k8temp.ko 2024-01-16T14:48:32.382 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lineage-pem.ko 2024-01-16T14:48:32.382 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm63.ko 2024-01-16T14:48:32.382 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm73.ko 2024-01-16T14:48:32.382 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm75.ko 2024-01-16T14:48:32.382 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm77.ko 2024-01-16T14:48:32.382 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm78.ko 2024-01-16T14:48:32.382 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm80.ko 2024-01-16T14:48:32.382 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm83.ko 2024-01-16T14:48:32.382 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm85.ko 2024-01-16T14:48:32.382 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm87.ko 2024-01-16T14:48:32.382 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm90.ko 2024-01-16T14:48:32.382 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm92.ko 2024-01-16T14:48:32.382 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm93.ko 2024-01-16T14:48:32.383 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm95234.ko 2024-01-16T14:48:32.383 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm95241.ko 2024-01-16T14:48:32.383 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm95245.ko 2024-01-16T14:48:32.383 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc2945.ko 2024-01-16T14:48:32.383 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc4151.ko 2024-01-16T14:48:32.383 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc4215.ko 2024-01-16T14:48:32.383 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc4222.ko 2024-01-16T14:48:32.383 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc4245.ko 2024-01-16T14:48:32.383 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc4260.ko 2024-01-16T14:48:32.383 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc4261.ko 2024-01-16T14:48:32.383 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max16065.ko 2024-01-16T14:48:32.383 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max1619.ko 2024-01-16T14:48:32.383 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max1668.ko 2024-01-16T14:48:32.383 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max197.ko 2024-01-16T14:48:32.383 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max6639.ko 2024-01-16T14:48:32.384 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max6650.ko 2024-01-16T14:48:32.384 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max6697.ko 2024-01-16T14:48:32.384 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/mcp3021.ko 2024-01-16T14:48:32.384 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/nct6683.ko 2024-01-16T14:48:32.384 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/nct6775-core.ko 2024-01-16T14:48:32.384 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/nct6775.ko 2024-01-16T14:48:32.384 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/nct7802.ko 2024-01-16T14:48:32.384 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pc87360.ko 2024-01-16T14:48:32.384 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pc87427.ko 2024-01-16T14:48:32.384 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pcf8591.ko 2024-01-16T14:48:32.384 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus 2024-01-16T14:48:32.384 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-01-16T14:48:32.384 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-01-16T14:48:32.384 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-01-16T14:48:32.384 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/max16064.ko 2024-01-16T14:48:32.385 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/max34440.ko 2024-01-16T14:48:32.385 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/max8688.ko 2024-01-16T14:48:32.385 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-01-16T14:48:32.385 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-01-16T14:48:32.385 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-01-16T14:48:32.385 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-01-16T14:48:32.385 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-01-16T14:48:32.385 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-01-16T14:48:32.385 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/powr1220.ko 2024-01-16T14:48:32.385 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/sch5627.ko 2024-01-16T14:48:32.385 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/sch5636.ko 2024-01-16T14:48:32.385 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/sch56xx-common.ko 2024-01-16T14:48:32.385 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/sht15.ko 2024-01-16T14:48:32.385 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/sht21.ko 2024-01-16T14:48:32.386 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/shtc1.ko 2024-01-16T14:48:32.386 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/sis5595.ko 2024-01-16T14:48:32.386 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/smsc47b397.ko 2024-01-16T14:48:32.386 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/smsc47m1.ko 2024-01-16T14:48:32.386 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/smsc47m192.ko 2024-01-16T14:48:32.386 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/thmc50.ko 2024-01-16T14:48:32.386 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/tmp102.ko 2024-01-16T14:48:32.386 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/tmp103.ko 2024-01-16T14:48:32.386 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/tmp401.ko 2024-01-16T14:48:32.386 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/tmp421.ko 2024-01-16T14:48:32.386 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/via-cputemp.ko 2024-01-16T14:48:32.386 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/via686a.ko 2024-01-16T14:48:32.386 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/vt1211.ko 2024-01-16T14:48:32.386 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/vt8231.ko 2024-01-16T14:48:32.386 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83627ehf.ko 2024-01-16T14:48:32.387 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83627hf.ko 2024-01-16T14:48:32.387 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83781d.ko 2024-01-16T14:48:32.387 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83791d.ko 2024-01-16T14:48:32.387 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83792d.ko 2024-01-16T14:48:32.387 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83793.ko 2024-01-16T14:48:32.387 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83795.ko 2024-01-16T14:48:32.387 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83l785ts.ko 2024-01-16T14:48:32.387 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83l786ng.ko 2024-01-16T14:48:32.387 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c 2024-01-16T14:48:32.387 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/algos 2024-01-16T14:48:32.387 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-01-16T14:48:32.387 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-01-16T14:48:32.387 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses 2024-01-16T14:48:32.387 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-01-16T14:48:32.387 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-01-16T14:48:32.388 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-01-16T14:48:32.388 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-01-16T14:48:32.388 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-01-16T14:48:32.388 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-01-16T14:48:32.388 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-01-16T14:48:32.388 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-01-16T14:48:32.388 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-i801.ko 2024-01-16T14:48:32.388 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-isch.ko 2024-01-16T14:48:32.388 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-01-16T14:48:32.388 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-01-16T14:48:32.388 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-01-16T14:48:32.388 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-parport.ko 2024-01-16T14:48:32.388 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-01-16T14:48:32.388 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-01-16T14:48:32.388 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-01-16T14:48:32.389 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-01-16T14:48:32.389 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-01-16T14:48:32.389 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-01-16T14:48:32.389 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-via.ko 2024-01-16T14:48:32.389 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-01-16T14:48:32.389 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-01-16T14:48:32.389 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/i2c-dev.ko 2024-01-16T14:48:32.389 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/i2c-mux.ko 2024-01-16T14:48:32.389 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-01-16T14:48:32.389 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/i2c-smbus.ko 2024-01-16T14:48:32.389 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/i2c-stub.ko 2024-01-16T14:48:32.389 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband 2024-01-16T14:48:32.389 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core 2024-01-16T14:48:32.389 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/ib_cm.ko 2024-01-16T14:48:32.389 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/ib_core.ko 2024-01-16T14:48:32.390 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/ib_umad.ko 2024-01-16T14:48:32.390 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-01-16T14:48:32.390 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/iw_cm.ko 2024-01-16T14:48:32.390 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/rdma_cm.ko 2024-01-16T14:48:32.390 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-01-16T14:48:32.390 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw 2024-01-16T14:48:32.390 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/cxgb4 2024-01-16T14:48:32.390 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-01-16T14:48:32.390 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/mlx4 2024-01-16T14:48:32.390 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-01-16T14:48:32.390 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/mlx5 2024-01-16T14:48:32.390 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-01-16T14:48:32.390 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/mthca 2024-01-16T14:48:32.390 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-01-16T14:48:32.391 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp 2024-01-16T14:48:32.391 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/ipoib 2024-01-16T14:48:32.391 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-01-16T14:48:32.391 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/iser 2024-01-16T14:48:32.391 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-01-16T14:48:32.391 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/isert 2024-01-16T14:48:32.391 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-01-16T14:48:32.391 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/srp 2024-01-16T14:48:32.391 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-01-16T14:48:32.391 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/srpt 2024-01-16T14:48:32.391 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-01-16T14:48:32.391 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input 2024-01-16T14:48:32.391 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/gameport 2024-01-16T14:48:32.391 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-01-16T14:48:32.391 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/gameport/fm801-gp.ko 2024-01-16T14:48:32.392 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/gameport/gameport.ko 2024-01-16T14:48:32.392 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/joydev.ko 2024-01-16T14:48:32.392 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard 2024-01-16T14:48:32.392 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/gpio_keys.ko 2024-01-16T14:48:32.392 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse 2024-01-16T14:48:32.392 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/appletouch.ko 2024-01-16T14:48:32.392 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/bcm5974.ko 2024-01-16T14:48:32.392 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/cyapatp.ko 2024-01-16T14:48:32.392 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/elan_i2c.ko 2024-01-16T14:48:32.392 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/sermouse.ko 2024-01-16T14:48:32.392 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-01-16T14:48:32.392 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/synaptics_usb.ko 2024-01-16T14:48:32.392 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/vsxxxaa.ko 2024-01-16T14:48:32.393 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/rmi4 2024-01-16T14:48:32.393 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/rmi4/rmi_core.ko 2024-01-16T14:48:32.393 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio 2024-01-16T14:48:32.393 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio/altera_ps2.ko 2024-01-16T14:48:32.393 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio/arc_ps2.ko 2024-01-16T14:48:32.393 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-01-16T14:48:32.393 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio/serio_raw.ko 2024-01-16T14:48:32.393 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/sparse-keymap.ko 2024-01-16T14:48:32.393 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds 2024-01-16T14:48:32.393 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-blinkm.ko 2024-01-16T14:48:32.393 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-lm3530.ko 2024-01-16T14:48:32.393 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-lp3944.ko 2024-01-16T14:48:32.393 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-ss4200.ko 2024-01-16T14:48:32.393 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger 2024-01-16T14:48:32.393 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-01-16T14:48:32.394 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-01-16T14:48:32.394 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-01-16T14:48:32.394 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-01-16T14:48:32.394 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-01-16T14:48:32.394 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-01-16T14:48:32.394 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-01-16T14:48:32.394 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-01-16T14:48:32.394 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-01-16T14:48:32.394 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md 2024-01-16T14:48:32.394 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/bcache 2024-01-16T14:48:32.394 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/bcache/bcache.ko 2024-01-16T14:48:32.394 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-bio-prison.ko 2024-01-16T14:48:32.394 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-cache-smq.ko 2024-01-16T14:48:32.394 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-cache.ko 2024-01-16T14:48:32.394 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-crypt.ko 2024-01-16T14:48:32.395 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-delay.ko 2024-01-16T14:48:32.395 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-flakey.ko 2024-01-16T14:48:32.395 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-log-userspace.ko 2024-01-16T14:48:32.395 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-multipath.ko 2024-01-16T14:48:32.395 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-queue-length.ko 2024-01-16T14:48:32.395 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-raid.ko 2024-01-16T14:48:32.395 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-round-robin.ko 2024-01-16T14:48:32.395 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-service-time.ko 2024-01-16T14:48:32.395 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-switch.ko 2024-01-16T14:48:32.395 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-thin-pool.ko 2024-01-16T14:48:32.395 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-verity.ko 2024-01-16T14:48:32.395 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/faulty.ko 2024-01-16T14:48:32.395 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/linear.ko 2024-01-16T14:48:32.395 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/multipath.ko 2024-01-16T14:48:32.395 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/persistent-data 2024-01-16T14:48:32.396 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-01-16T14:48:32.396 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/raid0.ko 2024-01-16T14:48:32.396 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/raid1.ko 2024-01-16T14:48:32.396 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/raid10.ko 2024-01-16T14:48:32.396 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/raid456.ko 2024-01-16T14:48:32.396 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/message 2024-01-16T14:48:32.396 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion 2024-01-16T14:48:32.396 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptbase.ko 2024-01-16T14:48:32.396 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptctl.ko 2024-01-16T14:48:32.396 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptfc.ko 2024-01-16T14:48:32.396 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptlan.ko 2024-01-16T14:48:32.396 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptsas.ko 2024-01-16T14:48:32.396 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptscsih.ko 2024-01-16T14:48:32.396 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptspi.ko 2024-01-16T14:48:32.396 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd 2024-01-16T14:48:32.397 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/lpc_ich.ko 2024-01-16T14:48:32.397 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/lpc_sch.ko 2024-01-16T14:48:32.397 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/mfd-core.ko 2024-01-16T14:48:32.397 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/sm501.ko 2024-01-16T14:48:32.397 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/viperboard.ko 2024-01-16T14:48:32.397 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/vx855.ko 2024-01-16T14:48:32.397 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/wl1273-core.ko 2024-01-16T14:48:32.397 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc 2024-01-16T14:48:32.397 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/altera-stapl 2024-01-16T14:48:32.397 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-01-16T14:48:32.397 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/apds9802als.ko 2024-01-16T14:48:32.397 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/apds990x.ko 2024-01-16T14:48:32.397 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/bh1770glc.ko 2024-01-16T14:48:32.397 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/cb710 2024-01-16T14:48:32.398 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/cb710/cb710.ko 2024-01-16T14:48:32.398 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/eeprom 2024-01-16T14:48:32.398 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/eeprom/at24.ko 2024-01-16T14:48:32.398 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-01-16T14:48:32.398 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/eeprom/max6875.ko 2024-01-16T14:48:32.398 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/enclosure.ko 2024-01-16T14:48:32.398 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/hpilo.ko 2024-01-16T14:48:32.398 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/isl29003.ko 2024-01-16T14:48:32.398 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/isl29020.ko 2024-01-16T14:48:32.398 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/lis3lv02d 2024-01-16T14:48:32.398 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-01-16T14:48:32.398 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-01-16T14:48:32.398 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/mei 2024-01-16T14:48:32.398 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/mei/mei-me.ko 2024-01-16T14:48:32.398 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/mei/mei-txe.ko 2024-01-16T14:48:32.399 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/mei/mei.ko 2024-01-16T14:48:32.399 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/sgi-gru 2024-01-16T14:48:32.399 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/sgi-gru/gru.ko 2024-01-16T14:48:32.399 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/sgi-xp 2024-01-16T14:48:32.399 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/sgi-xp/xp.ko 2024-01-16T14:48:32.399 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/sgi-xp/xpc.ko 2024-01-16T14:48:32.399 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-01-16T14:48:32.399 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/tifm_7xx1.ko 2024-01-16T14:48:32.399 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/tifm_core.ko 2024-01-16T14:48:32.399 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/tsl2550.ko 2024-01-16T14:48:32.399 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/vmw_balloon.ko 2024-01-16T14:48:32.399 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/vmw_vmci 2024-01-16T14:48:32.399 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-01-16T14:48:32.399 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net 2024-01-16T14:48:32.399 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/bonding 2024-01-16T14:48:32.400 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/bonding/bonding.ko 2024-01-16T14:48:32.400 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa 2024-01-16T14:48:32.400 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa/b53 2024-01-16T14:48:32.400 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa/b53/b53_common.ko 2024-01-16T14:48:32.400 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa/bcm-sf2.ko 2024-01-16T14:48:32.400 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa/mv88e6060.ko 2024-01-16T14:48:32.400 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa/mv88e6xxx 2024-01-16T14:48:32.400 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-01-16T14:48:32.400 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dummy.ko 2024-01-16T14:48:32.400 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/eql.ko 2024-01-16T14:48:32.400 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet 2024-01-16T14:48:32.400 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/3com 2024-01-16T14:48:32.400 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-01-16T14:48:32.401 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-01-16T14:48:32.401 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-01-16T14:48:32.401 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-01-16T14:48:32.401 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/8390 2024-01-16T14:48:32.401 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/8390/8390.ko 2024-01-16T14:48:32.401 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-01-16T14:48:32.401 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-01-16T14:48:32.401 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-01-16T14:48:32.401 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/adaptec 2024-01-16T14:48:32.401 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-01-16T14:48:32.401 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/agere 2024-01-16T14:48:32.401 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/agere/et131x.ko 2024-01-16T14:48:32.401 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/alteon 2024-01-16T14:48:32.401 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-01-16T14:48:32.401 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/altera 2024-01-16T14:48:32.402 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-01-16T14:48:32.402 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/amd 2024-01-16T14:48:32.402 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-01-16T14:48:32.402 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-01-16T14:48:32.402 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-01-16T14:48:32.402 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros 2024-01-16T14:48:32.402 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/alx 2024-01-16T14:48:32.402 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-01-16T14:48:32.402 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atl1c 2024-01-16T14:48:32.402 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-01-16T14:48:32.402 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atl1e 2024-01-16T14:48:32.402 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-01-16T14:48:32.402 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atlx 2024-01-16T14:48:32.402 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-01-16T14:48:32.402 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-01-16T14:48:32.403 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom 2024-01-16T14:48:32.403 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-01-16T14:48:32.403 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-01-16T14:48:32.403 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-01-16T14:48:32.403 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-01-16T14:48:32.403 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-01-16T14:48:32.403 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/genet 2024-01-16T14:48:32.403 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-01-16T14:48:32.403 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-01-16T14:48:32.403 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/brocade 2024-01-16T14:48:32.403 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/brocade/bna 2024-01-16T14:48:32.403 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-01-16T14:48:32.403 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio 2024-01-16T14:48:32.403 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb 2024-01-16T14:48:32.403 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-01-16T14:48:32.404 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-01-16T14:48:32.404 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-01-16T14:48:32.404 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-01-16T14:48:32.404 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-01-16T14:48:32.404 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-01-16T14:48:32.404 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-01-16T14:48:32.404 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-01-16T14:48:32.404 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-01-16T14:48:32.404 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/cisco 2024-01-16T14:48:32.404 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/cisco/enic 2024-01-16T14:48:32.404 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-01-16T14:48:32.404 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec 2024-01-16T14:48:32.404 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip 2024-01-16T14:48:32.404 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-01-16T14:48:32.405 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-01-16T14:48:32.405 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-01-16T14:48:32.405 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-01-16T14:48:32.405 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-01-16T14:48:32.405 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-01-16T14:48:32.405 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dlink 2024-01-16T14:48:32.405 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-01-16T14:48:32.405 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-01-16T14:48:32.405 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dnet.ko 2024-01-16T14:48:32.405 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/emulex 2024-01-16T14:48:32.405 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/emulex/benet 2024-01-16T14:48:32.405 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-01-16T14:48:32.405 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/ethoc.ko 2024-01-16T14:48:32.405 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/fealnx.ko 2024-01-16T14:48:32.405 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel 2024-01-16T14:48:32.406 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/e100.ko 2024-01-16T14:48:32.406 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/e1000 2024-01-16T14:48:32.406 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-01-16T14:48:32.406 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/e1000e 2024-01-16T14:48:32.406 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-01-16T14:48:32.406 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/fm10k 2024-01-16T14:48:32.406 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-01-16T14:48:32.406 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/i40e 2024-01-16T14:48:32.406 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-01-16T14:48:32.406 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/iavf 2024-01-16T14:48:32.406 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-01-16T14:48:32.406 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/igb 2024-01-16T14:48:32.406 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-01-16T14:48:32.406 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/igbvf 2024-01-16T14:48:32.406 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-01-16T14:48:32.407 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/ixgbe 2024-01-16T14:48:32.407 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-01-16T14:48:32.407 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/ixgbevf 2024-01-16T14:48:32.407 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-01-16T14:48:32.407 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/jme.ko 2024-01-16T14:48:32.407 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/marvell 2024-01-16T14:48:32.407 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-01-16T14:48:32.407 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/marvell/skge.ko 2024-01-16T14:48:32.407 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-01-16T14:48:32.407 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox 2024-01-16T14:48:32.407 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox/mlx4 2024-01-16T14:48:32.407 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-01-16T14:48:32.407 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-01-16T14:48:32.407 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox/mlx5 2024-01-16T14:48:32.407 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-01-16T14:48:32.408 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-01-16T14:48:32.408 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/micrel 2024-01-16T14:48:32.408 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-01-16T14:48:32.408 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/myricom 2024-01-16T14:48:32.408 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/myricom/myri10ge 2024-01-16T14:48:32.408 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-01-16T14:48:32.408 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/natsemi 2024-01-16T14:48:32.408 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-01-16T14:48:32.408 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-01-16T14:48:32.408 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/neterion 2024-01-16T14:48:32.408 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-01-16T14:48:32.408 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/nvidia 2024-01-16T14:48:32.408 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-01-16T14:48:32.408 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/packetengines 2024-01-16T14:48:32.409 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-01-16T14:48:32.409 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-01-16T14:48:32.409 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/qlogic 2024-01-16T14:48:32.409 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/qlogic/netxen 2024-01-16T14:48:32.409 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-01-16T14:48:32.409 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-01-16T14:48:32.409 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-01-16T14:48:32.409 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-01-16T14:48:32.409 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/rdc 2024-01-16T14:48:32.409 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-01-16T14:48:32.409 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/realtek 2024-01-16T14:48:32.410 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-01-16T14:48:32.410 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-01-16T14:48:32.410 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/realtek/atp.ko 2024-01-16T14:48:32.410 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-01-16T14:48:32.410 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/rocker 2024-01-16T14:48:32.410 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-01-16T14:48:32.410 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sfc 2024-01-16T14:48:32.410 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-01-16T14:48:32.411 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/silan 2024-01-16T14:48:32.411 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-01-16T14:48:32.411 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sis 2024-01-16T14:48:32.411 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sis/sis190.ko 2024-01-16T14:48:32.411 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sis/sis900.ko 2024-01-16T14:48:32.411 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/smsc 2024-01-16T14:48:32.411 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-01-16T14:48:32.411 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-01-16T14:48:32.412 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-01-16T14:48:32.412 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-01-16T14:48:32.412 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/stmicro 2024-01-16T14:48:32.412 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/stmicro/stmmac 2024-01-16T14:48:32.412 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-01-16T14:48:32.412 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-01-16T14:48:32.413 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sun 2024-01-16T14:48:32.413 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sun/cassini.ko 2024-01-16T14:48:32.413 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sun/niu.ko 2024-01-16T14:48:32.413 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sun/sungem.ko 2024-01-16T14:48:32.413 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-01-16T14:48:32.413 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/tehuti 2024-01-16T14:48:32.413 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-01-16T14:48:32.414 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/ti 2024-01-16T14:48:32.414 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/ti/tlan.ko 2024-01-16T14:48:32.414 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/via 2024-01-16T14:48:32.414 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-01-16T14:48:32.414 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-01-16T14:48:32.414 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/wiznet 2024-01-16T14:48:32.414 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-01-16T14:48:32.414 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-01-16T14:48:32.415 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/xircom 2024-01-16T14:48:32.415 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-01-16T14:48:32.415 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/geneve.ko 2024-01-16T14:48:32.415 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/hyperv 2024-01-16T14:48:32.415 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-01-16T14:48:32.415 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ieee802154 2024-01-16T14:48:32.415 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ieee802154/fakelb.ko 2024-01-16T14:48:32.415 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ifb.ko 2024-01-16T14:48:32.415 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ipvlan 2024-01-16T14:48:32.416 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ipvlan/ipvlan.ko 2024-01-16T14:48:32.416 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/macvlan.ko 2024-01-16T14:48:32.416 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/macvtap.ko 2024-01-16T14:48:32.416 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/mdio 2024-01-16T14:48:32.416 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/mdio.ko 2024-01-16T14:48:32.416 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-01-16T14:48:32.416 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-01-16T14:48:32.417 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/mdio/mdio-regmap.ko 2024-01-16T14:48:32.417 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/mii.ko 2024-01-16T14:48:32.417 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/net_failover.ko 2024-01-16T14:48:32.417 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/netconsole.ko 2024-01-16T14:48:32.417 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/nlmon.ko 2024-01-16T14:48:32.417 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/pcs 2024-01-16T14:48:32.418 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/pcs/pcs-lynx.ko 2024-01-16T14:48:32.418 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-01-16T14:48:32.418 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy 2024-01-16T14:48:32.418 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/amd.ko 2024-01-16T14:48:32.418 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-01-16T14:48:32.418 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-01-16T14:48:32.418 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/bcm7xxx.ko 2024-01-16T14:48:32.418 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/bcm87xx.ko 2024-01-16T14:48:32.418 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/broadcom.ko 2024-01-16T14:48:32.419 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/cicada.ko 2024-01-16T14:48:32.419 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/davicom.ko 2024-01-16T14:48:32.419 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/dp83640.ko 2024-01-16T14:48:32.419 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/et1011c.ko 2024-01-16T14:48:32.419 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/icplus.ko 2024-01-16T14:48:32.419 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/lxt.ko 2024-01-16T14:48:32.419 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/marvell.ko 2024-01-16T14:48:32.419 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/micrel.ko 2024-01-16T14:48:32.419 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/national.ko 2024-01-16T14:48:32.419 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/phylink.ko 2024-01-16T14:48:32.419 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/qsemi.ko 2024-01-16T14:48:32.419 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/realtek.ko 2024-01-16T14:48:32.419 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/smsc.ko 2024-01-16T14:48:32.419 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/ste10Xp.ko 2024-01-16T14:48:32.419 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/vitesse.ko 2024-01-16T14:48:32.420 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp 2024-01-16T14:48:32.420 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/bsd_comp.ko 2024-01-16T14:48:32.420 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/ppp_async.ko 2024-01-16T14:48:32.420 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/ppp_deflate.ko 2024-01-16T14:48:32.420 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/ppp_generic.ko 2024-01-16T14:48:32.420 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/ppp_mppe.ko 2024-01-16T14:48:32.420 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/ppp_synctty.ko 2024-01-16T14:48:32.420 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/pppoe.ko 2024-01-16T14:48:32.420 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/pppox.ko 2024-01-16T14:48:32.420 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/pptp.ko 2024-01-16T14:48:32.420 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/slip 2024-01-16T14:48:32.420 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/slip/slhc.ko 2024-01-16T14:48:32.420 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/slip/slip.ko 2024-01-16T14:48:32.420 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/sungem_phy.ko 2024-01-16T14:48:32.421 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/tap.ko 2024-01-16T14:48:32.421 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team 2024-01-16T14:48:32.421 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team/team.ko 2024-01-16T14:48:32.421 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team/team_mode_activebackup.ko 2024-01-16T14:48:32.421 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team/team_mode_broadcast.ko 2024-01-16T14:48:32.421 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-01-16T14:48:32.421 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team/team_mode_random.ko 2024-01-16T14:48:32.421 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-01-16T14:48:32.421 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/tun.ko 2024-01-16T14:48:32.421 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/veth.ko 2024-01-16T14:48:32.421 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/virtio_net.ko 2024-01-16T14:48:32.421 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/vmxnet3 2024-01-16T14:48:32.421 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-01-16T14:48:32.421 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/vxlan 2024-01-16T14:48:32.421 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/vxlan/vxlan.ko 2024-01-16T14:48:32.421 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/xen-netback 2024-01-16T14:48:32.422 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/xen-netback/xen-netback.ko 2024-01-16T14:48:32.422 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/xen-netfront.ko 2024-01-16T14:48:32.422 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ntb 2024-01-16T14:48:32.422 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ntb/ntb.ko 2024-01-16T14:48:32.422 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/nvme 2024-01-16T14:48:32.422 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/nvme/host 2024-01-16T14:48:32.422 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/nvme/host/nvme-core.ko 2024-01-16T14:48:32.422 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/nvme/host/nvme.ko 2024-01-16T14:48:32.422 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/parport 2024-01-16T14:48:32.422 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/parport/parport.ko 2024-01-16T14:48:32.422 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/parport/parport_cs.ko 2024-01-16T14:48:32.422 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/parport/parport_pc.ko 2024-01-16T14:48:32.422 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/parport/parport_serial.ko 2024-01-16T14:48:32.422 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pci 2024-01-16T14:48:32.422 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pci/hotplug 2024-01-16T14:48:32.422 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-01-16T14:48:32.422 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pci/pcie 2024-01-16T14:48:32.423 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pci/pcie/aer_inject.ko 2024-01-16T14:48:32.423 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pci/xen-pcifront.ko 2024-01-16T14:48:32.423 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pcmcia 2024-01-16T14:48:32.423 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pcmcia/i82092.ko 2024-01-16T14:48:32.423 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pcmcia/pd6729.ko 2024-01-16T14:48:32.423 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pcmcia/yenta_socket.ko 2024-01-16T14:48:32.423 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pinctrl 2024-01-16T14:48:32.423 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pinctrl/intel 2024-01-16T14:48:32.423 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-01-16T14:48:32.423 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform 2024-01-16T14:48:32.423 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86 2024-01-16T14:48:32.423 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/acer-wmi.ko 2024-01-16T14:48:32.423 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/acerhdf.ko 2024-01-16T14:48:32.423 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/apple-gmux.ko 2024-01-16T14:48:32.423 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/asus-laptop.ko 2024-01-16T14:48:32.423 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-01-16T14:48:32.424 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/asus-wmi.ko 2024-01-16T14:48:32.424 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/classmate-laptop.ko 2024-01-16T14:48:32.424 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-01-16T14:48:32.424 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-01-16T14:48:32.424 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-01-16T14:48:32.424 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-01-16T14:48:32.424 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/hdaps.ko 2024-01-16T14:48:32.424 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/intel 2024-01-16T14:48:32.424 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-01-16T14:48:32.424 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/intel_ips.ko 2024-01-16T14:48:32.424 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/msi-wmi.ko 2024-01-16T14:48:32.424 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/mxm-wmi.ko 2024-01-16T14:48:32.424 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-01-16T14:48:32.424 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/samsung-laptop.ko 2024-01-16T14:48:32.424 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/samsung-q10.ko 2024-01-16T14:48:32.425 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/topstar-laptop.ko 2024-01-16T14:48:32.425 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-01-16T14:48:32.425 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/toshiba_haps.ko 2024-01-16T14:48:32.425 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/wmi-bmof.ko 2024-01-16T14:48:32.425 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/wmi.ko 2024-01-16T14:48:32.425 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/powercap 2024-01-16T14:48:32.425 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/powercap/intel_rapl_common.ko 2024-01-16T14:48:32.425 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/powercap/intel_rapl_msr.ko 2024-01-16T14:48:32.425 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pps 2024-01-16T14:48:32.425 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pps/clients 2024-01-16T14:48:32.425 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pps/clients/pps-gpio.ko 2024-01-16T14:48:32.425 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pps/clients/pps-ldisc.ko 2024-01-16T14:48:32.425 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pps/clients/pps_parport.ko 2024-01-16T14:48:32.425 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pps/pps_core.ko 2024-01-16T14:48:32.425 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ptp 2024-01-16T14:48:32.426 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ptp/ptp.ko 2024-01-16T14:48:32.426 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ptp/ptp_kvm.ko 2024-01-16T14:48:32.426 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc 2024-01-16T14:48:32.426 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-bq32k.ko 2024-01-16T14:48:32.426 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1286.ko 2024-01-16T14:48:32.426 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1307.ko 2024-01-16T14:48:32.426 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1374.ko 2024-01-16T14:48:32.426 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1511.ko 2024-01-16T14:48:32.426 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1553.ko 2024-01-16T14:48:32.426 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1672.ko 2024-01-16T14:48:32.426 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1742.ko 2024-01-16T14:48:32.426 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds2404.ko 2024-01-16T14:48:32.426 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds3232.ko 2024-01-16T14:48:32.426 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-em3027.ko 2024-01-16T14:48:32.426 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-fm3130.ko 2024-01-16T14:48:32.426 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-isl12022.ko 2024-01-16T14:48:32.426 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-isl1208.ko 2024-01-16T14:48:32.427 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-m41t80.ko 2024-01-16T14:48:32.427 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-m48t35.ko 2024-01-16T14:48:32.427 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-m48t59.ko 2024-01-16T14:48:32.427 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-max6900.ko 2024-01-16T14:48:32.427 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-msm6242.ko 2024-01-16T14:48:32.427 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-pcf2127.ko 2024-01-16T14:48:32.427 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-pcf85063.ko 2024-01-16T14:48:32.427 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-pcf8523.ko 2024-01-16T14:48:32.427 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-pcf8563.ko 2024-01-16T14:48:32.427 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-pcf8583.ko 2024-01-16T14:48:32.427 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-rp5c01.ko 2024-01-16T14:48:32.427 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-rs5c372.ko 2024-01-16T14:48:32.427 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-01-16T14:48:32.427 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-rx8025.ko 2024-01-16T14:48:32.427 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-rx8581.ko 2024-01-16T14:48:32.427 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-01-16T14:48:32.428 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-x1205.ko 2024-01-16T14:48:32.428 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi 2024-01-16T14:48:32.428 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/3w-9xxx.ko 2024-01-16T14:48:32.428 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/3w-sas.ko 2024-01-16T14:48:32.428 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/3w-xxxx.ko 2024-01-16T14:48:32.428 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/BusLogic.ko 2024-01-16T14:48:32.428 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/a100u2w.ko 2024-01-16T14:48:32.428 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/aacraid 2024-01-16T14:48:32.428 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/aacraid/aacraid.ko 2024-01-16T14:48:32.428 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/advansys.ko 2024-01-16T14:48:32.428 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/aic7xxx 2024-01-16T14:48:32.428 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-01-16T14:48:32.428 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-01-16T14:48:32.428 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/am53c974.ko 2024-01-16T14:48:32.429 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/arcmsr 2024-01-16T14:48:32.429 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-01-16T14:48:32.429 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/atp870u.ko 2024-01-16T14:48:32.429 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/be2iscsi 2024-01-16T14:48:32.429 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-01-16T14:48:32.429 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/bfa 2024-01-16T14:48:32.429 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/bfa/bfa.ko 2024-01-16T14:48:32.429 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/bnx2fc 2024-01-16T14:48:32.429 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-01-16T14:48:32.429 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/bnx2i 2024-01-16T14:48:32.429 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-01-16T14:48:32.429 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/ch.ko 2024-01-16T14:48:32.429 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/csiostor 2024-01-16T14:48:32.430 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/csiostor/csiostor.ko 2024-01-16T14:48:32.430 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/cxgbi 2024-01-16T14:48:32.430 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/cxgbi/cxgb3i 2024-01-16T14:48:32.430 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-01-16T14:48:32.430 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/cxgbi/cxgb4i 2024-01-16T14:48:32.430 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-01-16T14:48:32.430 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-01-16T14:48:32.430 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/dc395x.ko 2024-01-16T14:48:32.430 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/device_handler 2024-01-16T14:48:32.430 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-01-16T14:48:32.430 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-01-16T14:48:32.430 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-01-16T14:48:32.430 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-01-16T14:48:32.430 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/dmx3191d.ko 2024-01-16T14:48:32.431 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/esas2r 2024-01-16T14:48:32.431 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/esas2r/esas2r.ko 2024-01-16T14:48:32.431 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/esp_scsi.ko 2024-01-16T14:48:32.431 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/fcoe 2024-01-16T14:48:32.431 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/fcoe/fcoe.ko 2024-01-16T14:48:32.431 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-01-16T14:48:32.431 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/fnic 2024-01-16T14:48:32.431 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/fnic/fnic.ko 2024-01-16T14:48:32.431 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/hpsa.ko 2024-01-16T14:48:32.431 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/hptiop.ko 2024-01-16T14:48:32.431 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/hv_storvsc.ko 2024-01-16T14:48:32.431 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/initio.ko 2024-01-16T14:48:32.431 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/ipr.ko 2024-01-16T14:48:32.431 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/ips.ko 2024-01-16T14:48:32.431 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/isci 2024-01-16T14:48:32.431 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/isci/isci.ko 2024-01-16T14:48:32.432 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-01-16T14:48:32.432 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/iscsi_tcp.ko 2024-01-16T14:48:32.432 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/libfc 2024-01-16T14:48:32.432 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/libfc/libfc.ko 2024-01-16T14:48:32.432 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/libiscsi.ko 2024-01-16T14:48:32.432 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/libiscsi_tcp.ko 2024-01-16T14:48:32.432 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/libsas 2024-01-16T14:48:32.432 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/libsas/libsas.ko 2024-01-16T14:48:32.432 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/lpfc 2024-01-16T14:48:32.432 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/lpfc/lpfc.ko 2024-01-16T14:48:32.432 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/megaraid 2024-01-16T14:48:32.432 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/megaraid.ko 2024-01-16T14:48:32.432 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-01-16T14:48:32.433 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-01-16T14:48:32.433 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-01-16T14:48:32.433 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/mpt3sas 2024-01-16T14:48:32.433 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-01-16T14:48:32.433 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/mvsas 2024-01-16T14:48:32.433 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/mvsas/mvsas.ko 2024-01-16T14:48:32.433 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/mvumi.ko 2024-01-16T14:48:32.433 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/pm8001 2024-01-16T14:48:32.433 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-01-16T14:48:32.433 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/pmcraid.ko 2024-01-16T14:48:32.433 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/qla1280.ko 2024-01-16T14:48:32.433 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/qla2xxx 2024-01-16T14:48:32.433 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-01-16T14:48:32.433 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-01-16T14:48:32.433 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/qla4xxx 2024-01-16T14:48:32.434 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-01-16T14:48:32.434 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/raid_class.ko 2024-01-16T14:48:32.434 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/scsi_debug.ko 2024-01-16T14:48:32.434 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/scsi_transport_fc.ko 2024-01-16T14:48:32.434 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-01-16T14:48:32.434 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/scsi_transport_sas.ko 2024-01-16T14:48:32.434 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/scsi_transport_spi.ko 2024-01-16T14:48:32.434 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/scsi_transport_srp.ko 2024-01-16T14:48:32.434 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/ses.ko 2024-01-16T14:48:32.434 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/st.ko 2024-01-16T14:48:32.434 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/stex.ko 2024-01-16T14:48:32.434 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/sym53c8xx_2 2024-01-16T14:48:32.434 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-01-16T14:48:32.434 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/virtio_scsi.ko 2024-01-16T14:48:32.435 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/vmw_pvscsi.ko 2024-01-16T14:48:32.435 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/wd719x.ko 2024-01-16T14:48:32.435 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ssb 2024-01-16T14:48:32.435 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ssb/ssb.ko 2024-01-16T14:48:32.435 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target 2024-01-16T14:48:32.435 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/iscsi 2024-01-16T14:48:32.435 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-01-16T14:48:32.435 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/loopback 2024-01-16T14:48:32.435 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/loopback/tcm_loop.ko 2024-01-16T14:48:32.435 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/target_core_file.ko 2024-01-16T14:48:32.435 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/target_core_iblock.ko 2024-01-16T14:48:32.435 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/target_core_mod.ko 2024-01-16T14:48:32.435 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/target_core_pscsi.ko 2024-01-16T14:48:32.435 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/target_core_user.ko 2024-01-16T14:48:32.435 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/tcm_fc 2024-01-16T14:48:32.436 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-01-16T14:48:32.436 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal 2024-01-16T14:48:32.436 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel 2024-01-16T14:48:32.436 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal 2024-01-16T14:48:32.436 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-01-16T14:48:32.436 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-01-16T14:48:32.436 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-01-16T14:48:32.436 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-01-16T14:48:32.436 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-01-16T14:48:32.436 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-01-16T14:48:32.436 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-01-16T14:48:32.436 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-01-16T14:48:32.436 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-01-16T14:48:32.436 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-01-16T14:48:32.437 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-01-16T14:48:32.437 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-01-16T14:48:32.437 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-01-16T14:48:32.437 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-01-16T14:48:32.437 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-01-16T14:48:32.437 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-01-16T14:48:32.437 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-01-16T14:48:32.437 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-01-16T14:48:32.437 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty 2024-01-16T14:48:32.437 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/ipwireless 2024-01-16T14:48:32.437 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-01-16T14:48:32.437 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/n_gsm.ko 2024-01-16T14:48:32.437 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/n_hdlc.ko 2024-01-16T14:48:32.437 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/nozomi.ko 2024-01-16T14:48:32.438 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial 2024-01-16T14:48:32.438 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/8250 2024-01-16T14:48:32.438 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-01-16T14:48:32.438 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/arc_uart.ko 2024-01-16T14:48:32.438 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/jsm 2024-01-16T14:48:32.438 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/jsm/jsm.ko 2024-01-16T14:48:32.438 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ufs 2024-01-16T14:48:32.438 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ufs/core 2024-01-16T14:48:32.438 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ufs/core/ufshcd-core.ko 2024-01-16T14:48:32.438 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ufs/host 2024-01-16T14:48:32.438 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-01-16T14:48:32.438 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio 2024-01-16T14:48:32.438 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/uio.ko 2024-01-16T14:48:32.438 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/uio_aec.ko 2024-01-16T14:48:32.438 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/uio_cif.ko 2024-01-16T14:48:32.439 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/uio_pci_generic.ko 2024-01-16T14:48:32.439 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/uio_sercos3.ko 2024-01-16T14:48:32.439 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb 2024-01-16T14:48:32.439 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/class 2024-01-16T14:48:32.439 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/class/cdc-acm.ko 2024-01-16T14:48:32.439 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/class/cdc-wdm.ko 2024-01-16T14:48:32.439 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/class/usblp.ko 2024-01-16T14:48:32.439 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/class/usbtmc.ko 2024-01-16T14:48:32.439 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/host 2024-01-16T14:48:32.439 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/host/sl811-hcd.ko 2024-01-16T14:48:32.439 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/image 2024-01-16T14:48:32.439 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/image/mdc800.ko 2024-01-16T14:48:32.439 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/image/microtek.ko 2024-01-16T14:48:32.439 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc 2024-01-16T14:48:32.439 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/adutux.ko 2024-01-16T14:48:32.440 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/appledisplay.ko 2024-01-16T14:48:32.440 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/emi26.ko 2024-01-16T14:48:32.440 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/emi62.ko 2024-01-16T14:48:32.440 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/ezusb.ko 2024-01-16T14:48:32.440 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/idmouse.ko 2024-01-16T14:48:32.440 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/iowarrior.ko 2024-01-16T14:48:32.440 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/isight_firmware.ko 2024-01-16T14:48:32.440 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/ldusb.ko 2024-01-16T14:48:32.440 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/legousbtower.ko 2024-01-16T14:48:32.440 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/sisusbvga 2024-01-16T14:48:32.440 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-01-16T14:48:32.440 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/trancevibrator.ko 2024-01-16T14:48:32.440 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/usb3503.ko 2024-01-16T14:48:32.440 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/usblcd.ko 2024-01-16T14:48:32.441 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/usbsevseg.ko 2024-01-16T14:48:32.441 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/uss720.ko 2024-01-16T14:48:32.441 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/yurex.ko 2024-01-16T14:48:32.441 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/phy 2024-01-16T14:48:32.441 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/phy/phy-generic.ko 2024-01-16T14:48:32.441 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial 2024-01-16T14:48:32.441 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/aircable.ko 2024-01-16T14:48:32.441 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ark3116.ko 2024-01-16T14:48:32.441 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/belkin_sa.ko 2024-01-16T14:48:32.441 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ch341.ko 2024-01-16T14:48:32.441 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/cp210x.ko 2024-01-16T14:48:32.441 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/cyberjack.ko 2024-01-16T14:48:32.441 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/cypress_m8.ko 2024-01-16T14:48:32.441 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/digi_acceleport.ko 2024-01-16T14:48:32.442 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/empeg.ko 2024-01-16T14:48:32.442 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ftdi_sio.ko 2024-01-16T14:48:32.442 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/garmin_gps.ko 2024-01-16T14:48:32.442 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/io_edgeport.ko 2024-01-16T14:48:32.442 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/io_ti.ko 2024-01-16T14:48:32.442 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ipaq.ko 2024-01-16T14:48:32.442 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ipw.ko 2024-01-16T14:48:32.442 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ir-usb.ko 2024-01-16T14:48:32.442 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-01-16T14:48:32.442 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/keyspan.ko 2024-01-16T14:48:32.442 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/keyspan_pda.ko 2024-01-16T14:48:32.442 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/kl5kusb105.ko 2024-01-16T14:48:32.442 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/kobil_sct.ko 2024-01-16T14:48:32.442 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/mct_u232.ko 2024-01-16T14:48:32.442 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/mos7720.ko 2024-01-16T14:48:32.443 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/mos7840.ko 2024-01-16T14:48:32.443 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/navman.ko 2024-01-16T14:48:32.443 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/omninet.ko 2024-01-16T14:48:32.443 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/opticon.ko 2024-01-16T14:48:32.443 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/option.ko 2024-01-16T14:48:32.443 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/oti6858.ko 2024-01-16T14:48:32.443 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/pl2303.ko 2024-01-16T14:48:32.443 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/qcaux.ko 2024-01-16T14:48:32.443 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/qcserial.ko 2024-01-16T14:48:32.443 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/quatech2.ko 2024-01-16T14:48:32.443 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/safe_serial.ko 2024-01-16T14:48:32.443 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/sierra.ko 2024-01-16T14:48:32.443 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/spcp8x5.ko 2024-01-16T14:48:32.443 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ssu100.ko 2024-01-16T14:48:32.443 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/symbolserial.ko 2024-01-16T14:48:32.444 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-01-16T14:48:32.444 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-01-16T14:48:32.444 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/usb_debug.ko 2024-01-16T14:48:32.444 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/usb_wwan.ko 2024-01-16T14:48:32.444 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/visor.ko 2024-01-16T14:48:32.444 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/whiteheat.ko 2024-01-16T14:48:32.444 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/xsens_mt.ko 2024-01-16T14:48:32.444 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage 2024-01-16T14:48:32.444 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/uas.ko 2024-01-16T14:48:32.444 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-alauda.ko 2024-01-16T14:48:32.444 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-cypress.ko 2024-01-16T14:48:32.444 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-datafab.ko 2024-01-16T14:48:32.444 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-01-16T14:48:32.444 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-freecom.ko 2024-01-16T14:48:32.445 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-isd200.ko 2024-01-16T14:48:32.445 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-01-16T14:48:32.445 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-karma.ko 2024-01-16T14:48:32.445 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-onetouch.ko 2024-01-16T14:48:32.445 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-realtek.ko 2024-01-16T14:48:32.445 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-sddr09.ko 2024-01-16T14:48:32.445 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-sddr55.ko 2024-01-16T14:48:32.445 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-usbat.ko 2024-01-16T14:48:32.445 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/usb-storage.ko 2024-01-16T14:48:32.445 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/usbip 2024-01-16T14:48:32.445 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/usbip/usbip-core.ko 2024-01-16T14:48:32.445 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/usbip/usbip-host.ko 2024-01-16T14:48:32.445 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-01-16T14:48:32.445 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vfio 2024-01-16T14:48:32.445 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vfio/pci 2024-01-16T14:48:32.446 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-01-16T14:48:32.446 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vfio/pci/vfio-pci.ko 2024-01-16T14:48:32.446 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vfio/vfio.ko 2024-01-16T14:48:32.446 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-01-16T14:48:32.446 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vhost 2024-01-16T14:48:32.446 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vhost/vhost.ko 2024-01-16T14:48:32.446 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vhost/vhost_iotlb.ko 2024-01-16T14:48:32.446 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vhost/vhost_net.ko 2024-01-16T14:48:32.446 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vhost/vhost_scsi.ko 2024-01-16T14:48:32.446 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/video 2024-01-16T14:48:32.446 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight 2024-01-16T14:48:32.446 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/apple_bl.ko 2024-01-16T14:48:32.446 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/lcd.ko 2024-01-16T14:48:32.446 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/platform_lcd.ko 2024-01-16T14:48:32.446 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/virtio 2024-01-16T14:48:32.447 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/virtio/virtio.ko 2024-01-16T14:48:32.447 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/virtio/virtio_balloon.ko 2024-01-16T14:48:32.447 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/virtio/virtio_mmio.ko 2024-01-16T14:48:32.447 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/virtio/virtio_pci.ko 2024-01-16T14:48:32.447 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-01-16T14:48:32.447 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-01-16T14:48:32.447 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/virtio/virtio_ring.ko 2024-01-16T14:48:32.447 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1 2024-01-16T14:48:32.447 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/masters 2024-01-16T14:48:32.447 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/masters/ds2482.ko 2024-01-16T14:48:32.447 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/masters/ds2490.ko 2024-01-16T14:48:32.447 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves 2024-01-16T14:48:32.447 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-01-16T14:48:32.447 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-01-16T14:48:32.447 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-01-16T14:48:32.447 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-01-16T14:48:32.448 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-01-16T14:48:32.448 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-01-16T14:48:32.448 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-01-16T14:48:32.448 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-01-16T14:48:32.448 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-01-16T14:48:32.448 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_smem.ko 2024-01-16T14:48:32.448 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_therm.ko 2024-01-16T14:48:32.448 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/wire.ko 2024-01-16T14:48:32.448 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog 2024-01-16T14:48:32.448 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/alim1535_wdt.ko 2024-01-16T14:48:32.448 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/alim7101_wdt.ko 2024-01-16T14:48:32.448 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/f71808e_wdt.ko 2024-01-16T14:48:32.448 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/hpwdt.ko 2024-01-16T14:48:32.448 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/i6300esb.ko 2024-01-16T14:48:32.449 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-01-16T14:48:32.449 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/iTCO_wdt.ko 2024-01-16T14:48:32.449 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/ib700wdt.ko 2024-01-16T14:48:32.449 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/ibmasr.ko 2024-01-16T14:48:32.449 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-01-16T14:48:32.449 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/it8712f_wdt.ko 2024-01-16T14:48:32.449 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/it87_wdt.ko 2024-01-16T14:48:32.449 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/machzwd.ko 2024-01-16T14:48:32.449 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/nv_tco.ko 2024-01-16T14:48:32.449 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/pcwd_pci.ko 2024-01-16T14:48:32.449 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/pcwd_usb.ko 2024-01-16T14:48:32.449 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-01-16T14:48:32.449 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/sch311x_wdt.ko 2024-01-16T14:48:32.449 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/softdog.ko 2024-01-16T14:48:32.449 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/sp5100_tco.ko 2024-01-16T14:48:32.450 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/via_wdt.ko 2024-01-16T14:48:32.450 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-01-16T14:48:32.450 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/w83877f_wdt.ko 2024-01-16T14:48:32.450 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/w83977f_wdt.ko 2024-01-16T14:48:32.450 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/wdt_pci.ko 2024-01-16T14:48:32.450 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/xen_wdt.ko 2024-01-16T14:48:32.450 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen 2024-01-16T14:48:32.450 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-acpi-processor.ko 2024-01-16T14:48:32.450 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-evtchn.ko 2024-01-16T14:48:32.450 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-gntalloc.ko 2024-01-16T14:48:32.450 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-gntdev.ko 2024-01-16T14:48:32.450 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-pciback 2024-01-16T14:48:32.450 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-01-16T14:48:32.450 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-privcmd.ko 2024-01-16T14:48:32.450 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xenfs 2024-01-16T14:48:32.451 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xenfs/xenfs.ko 2024-01-16T14:48:32.451 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs 2024-01-16T14:48:32.451 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/9p 2024-01-16T14:48:32.451 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/9p/9p.ko 2024-01-16T14:48:32.451 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/affs 2024-01-16T14:48:32.451 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/affs/affs.ko 2024-01-16T14:48:32.451 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/befs 2024-01-16T14:48:32.451 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/befs/befs.ko 2024-01-16T14:48:32.451 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/binfmt_misc.ko 2024-01-16T14:48:32.451 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/btrfs 2024-01-16T14:48:32.451 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/btrfs/btrfs.ko 2024-01-16T14:48:32.451 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/cachefiles 2024-01-16T14:48:32.451 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/cachefiles/cachefiles.ko 2024-01-16T14:48:32.451 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ceph 2024-01-16T14:48:32.452 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ceph/ceph.ko 2024-01-16T14:48:32.452 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/coda 2024-01-16T14:48:32.452 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/coda/coda.ko 2024-01-16T14:48:32.452 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/cramfs 2024-01-16T14:48:32.452 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/cramfs/cramfs.ko 2024-01-16T14:48:32.452 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/dlm 2024-01-16T14:48:32.452 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/dlm/dlm.ko 2024-01-16T14:48:32.452 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ecryptfs 2024-01-16T14:48:32.452 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ecryptfs/ecryptfs.ko 2024-01-16T14:48:32.452 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/f2fs 2024-01-16T14:48:32.452 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/f2fs/f2fs.ko 2024-01-16T14:48:32.452 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/fat 2024-01-16T14:48:32.452 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/fat/fat.ko 2024-01-16T14:48:32.452 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/fat/msdos.ko 2024-01-16T14:48:32.453 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/fat/vfat.ko 2024-01-16T14:48:32.453 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/fscache 2024-01-16T14:48:32.453 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/fscache/fscache.ko 2024-01-16T14:48:32.453 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/fuse 2024-01-16T14:48:32.453 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/fuse/cuse.ko 2024-01-16T14:48:32.453 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/fuse/fuse.ko 2024-01-16T14:48:32.453 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/gfs2 2024-01-16T14:48:32.453 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/gfs2/gfs2.ko 2024-01-16T14:48:32.453 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/hfs 2024-01-16T14:48:32.453 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/hfs/hfs.ko 2024-01-16T14:48:32.453 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/hfsplus 2024-01-16T14:48:32.453 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/hfsplus/hfsplus.ko 2024-01-16T14:48:32.453 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/isofs 2024-01-16T14:48:32.453 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/isofs/isofs.ko 2024-01-16T14:48:32.454 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/jfs 2024-01-16T14:48:32.454 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/jfs/jfs.ko 2024-01-16T14:48:32.454 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/lockd 2024-01-16T14:48:32.454 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/lockd/lockd.ko 2024-01-16T14:48:32.454 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/minix 2024-01-16T14:48:32.454 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/minix/minix.ko 2024-01-16T14:48:32.454 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/netfs 2024-01-16T14:48:32.454 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/netfs/netfs.ko 2024-01-16T14:48:32.454 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs 2024-01-16T14:48:32.454 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/blocklayout 2024-01-16T14:48:32.454 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-01-16T14:48:32.454 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/filelayout 2024-01-16T14:48:32.454 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-01-16T14:48:32.454 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/flexfilelayout 2024-01-16T14:48:32.454 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-01-16T14:48:32.454 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/nfs.ko 2024-01-16T14:48:32.455 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/nfsv3.ko 2024-01-16T14:48:32.455 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/nfsv4.ko 2024-01-16T14:48:32.455 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs_common 2024-01-16T14:48:32.455 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs_common/grace.ko 2024-01-16T14:48:32.455 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs_common/nfs_acl.ko 2024-01-16T14:48:32.455 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfsd 2024-01-16T14:48:32.455 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfsd/nfsd.ko 2024-01-16T14:48:32.455 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nilfs2 2024-01-16T14:48:32.455 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nilfs2/nilfs2.ko 2024-01-16T14:48:32.455 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls 2024-01-16T14:48:32.455 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-celtic.ko 2024-01-16T14:48:32.455 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-centeuro.ko 2024-01-16T14:48:32.455 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-croatian.ko 2024-01-16T14:48:32.455 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-cyrillic.ko 2024-01-16T14:48:32.455 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-gaelic.ko 2024-01-16T14:48:32.456 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-greek.ko 2024-01-16T14:48:32.456 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-iceland.ko 2024-01-16T14:48:32.456 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-inuit.ko 2024-01-16T14:48:32.456 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-roman.ko 2024-01-16T14:48:32.456 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-romanian.ko 2024-01-16T14:48:32.456 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-turkish.ko 2024-01-16T14:48:32.456 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp1250.ko 2024-01-16T14:48:32.456 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp1251.ko 2024-01-16T14:48:32.456 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp1255.ko 2024-01-16T14:48:32.456 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp737.ko 2024-01-16T14:48:32.456 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp775.ko 2024-01-16T14:48:32.456 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp850.ko 2024-01-16T14:48:32.456 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp852.ko 2024-01-16T14:48:32.456 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp855.ko 2024-01-16T14:48:32.457 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp857.ko 2024-01-16T14:48:32.457 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp860.ko 2024-01-16T14:48:32.457 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp861.ko 2024-01-16T14:48:32.457 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp862.ko 2024-01-16T14:48:32.457 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp863.ko 2024-01-16T14:48:32.457 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp864.ko 2024-01-16T14:48:32.457 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp865.ko 2024-01-16T14:48:32.457 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp866.ko 2024-01-16T14:48:32.457 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp869.ko 2024-01-16T14:48:32.457 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp874.ko 2024-01-16T14:48:32.457 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp932.ko 2024-01-16T14:48:32.457 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp936.ko 2024-01-16T14:48:32.457 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp949.ko 2024-01-16T14:48:32.457 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp950.ko 2024-01-16T14:48:32.457 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_euc-jp.ko 2024-01-16T14:48:32.458 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-1.ko 2024-01-16T14:48:32.458 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-13.ko 2024-01-16T14:48:32.458 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-14.ko 2024-01-16T14:48:32.458 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-15.ko 2024-01-16T14:48:32.458 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-2.ko 2024-01-16T14:48:32.458 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-3.ko 2024-01-16T14:48:32.458 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-4.ko 2024-01-16T14:48:32.458 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-5.ko 2024-01-16T14:48:32.458 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-6.ko 2024-01-16T14:48:32.458 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-7.ko 2024-01-16T14:48:32.458 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-9.ko 2024-01-16T14:48:32.458 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_koi8-r.ko 2024-01-16T14:48:32.458 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_koi8-ru.ko 2024-01-16T14:48:32.458 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_koi8-u.ko 2024-01-16T14:48:32.458 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_ucs2_utils.ko 2024-01-16T14:48:32.458 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_utf8.ko 2024-01-16T14:48:32.459 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2 2024-01-16T14:48:32.459 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/cluster 2024-01-16T14:48:32.459 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-01-16T14:48:32.459 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/dlm 2024-01-16T14:48:32.459 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-01-16T14:48:32.459 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/dlmfs 2024-01-16T14:48:32.459 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-01-16T14:48:32.459 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/ocfs2.ko 2024-01-16T14:48:32.459 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-01-16T14:48:32.459 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-01-16T14:48:32.459 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-01-16T14:48:32.459 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/overlayfs 2024-01-16T14:48:32.459 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/overlayfs/overlay.ko 2024-01-16T14:48:32.459 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/pstore 2024-01-16T14:48:32.459 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/pstore/ramoops.ko 2024-01-16T14:48:32.460 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/reiserfs 2024-01-16T14:48:32.460 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/reiserfs/reiserfs.ko 2024-01-16T14:48:32.460 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/romfs 2024-01-16T14:48:32.460 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/romfs/romfs.ko 2024-01-16T14:48:32.460 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/smb 2024-01-16T14:48:32.460 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/smb/client 2024-01-16T14:48:32.460 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/smb/client/cifs.ko 2024-01-16T14:48:32.460 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/smb/common 2024-01-16T14:48:32.460 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/smb/common/cifs_arc4.ko 2024-01-16T14:48:32.460 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/smb/common/cifs_md4.ko 2024-01-16T14:48:32.460 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/squashfs 2024-01-16T14:48:32.460 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/squashfs/squashfs.ko 2024-01-16T14:48:32.460 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/sysv 2024-01-16T14:48:32.460 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/sysv/sysv.ko 2024-01-16T14:48:32.461 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/udf 2024-01-16T14:48:32.461 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/udf/udf.ko 2024-01-16T14:48:32.461 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ufs 2024-01-16T14:48:32.461 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ufs/ufs.ko 2024-01-16T14:48:32.461 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/xfs 2024-01-16T14:48:32.461 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/xfs/xfs.ko 2024-01-16T14:48:32.461 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/kernel 2024-01-16T14:48:32.461 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/kernel/rcu 2024-01-16T14:48:32.461 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/kernel/rcu/rcutorture.ko 2024-01-16T14:48:32.461 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/kernel/torture.ko 2024-01-16T14:48:32.461 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/kernel/trace 2024-01-16T14:48:32.461 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-01-16T14:48:32.461 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib 2024-01-16T14:48:32.461 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/asn1_encoder.ko 2024-01-16T14:48:32.461 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/crc-itu-t.ko 2024-01-16T14:48:32.462 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/crc8.ko 2024-01-16T14:48:32.462 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/crypto 2024-01-16T14:48:32.462 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/crypto/libarc4.ko 2024-01-16T14:48:32.462 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/crypto/libdes.ko 2024-01-16T14:48:32.462 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/libcrc32c.ko 2024-01-16T14:48:32.462 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/lru_cache.ko 2024-01-16T14:48:32.462 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/lz4 2024-01-16T14:48:32.462 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/lz4/lz4_compress.ko 2024-01-16T14:48:32.462 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/lz4/lz4hc_compress.ko 2024-01-16T14:48:32.462 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/math 2024-01-16T14:48:32.462 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/math/cordic.ko 2024-01-16T14:48:32.462 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/raid6 2024-01-16T14:48:32.462 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/raid6/raid6_pq.ko 2024-01-16T14:48:32.462 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/reed_solomon 2024-01-16T14:48:32.462 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/reed_solomon/reed_solomon.ko 2024-01-16T14:48:32.462 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/ts_bm.ko 2024-01-16T14:48:32.463 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/ts_fsm.ko 2024-01-16T14:48:32.463 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/ts_kmp.ko 2024-01-16T14:48:32.463 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/zstd 2024-01-16T14:48:32.463 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/zstd/zstd_compress.ko 2024-01-16T14:48:32.463 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/mm 2024-01-16T14:48:32.463 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/mm/hwpoison-inject.ko 2024-01-16T14:48:32.463 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net 2024-01-16T14:48:32.463 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan 2024-01-16T14:48:32.463 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/6lowpan.ko 2024-01-16T14:48:32.463 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_dest.ko 2024-01-16T14:48:32.463 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_fragment.ko 2024-01-16T14:48:32.463 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_hop.ko 2024-01-16T14:48:32.463 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_ipv6.ko 2024-01-16T14:48:32.463 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_mobility.ko 2024-01-16T14:48:32.463 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_routing.ko 2024-01-16T14:48:32.463 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_udp.ko 2024-01-16T14:48:32.464 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/802 2024-01-16T14:48:32.464 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/802/garp.ko 2024-01-16T14:48:32.464 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/802/mrp.ko 2024-01-16T14:48:32.464 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/802/p8022.ko 2024-01-16T14:48:32.464 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/802/psnap.ko 2024-01-16T14:48:32.464 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/802/stp.ko 2024-01-16T14:48:32.464 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/8021q 2024-01-16T14:48:32.464 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/8021q/8021q.ko 2024-01-16T14:48:32.464 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/9p 2024-01-16T14:48:32.464 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/9p/9pnet.ko 2024-01-16T14:48:32.464 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/9p/9pnet_fd.ko 2024-01-16T14:48:32.465 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/9p/9pnet_rdma.ko 2024-01-16T14:48:32.465 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/9p/9pnet_virtio.ko 2024-01-16T14:48:32.465 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/batman-adv 2024-01-16T14:48:32.465 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/batman-adv/batman-adv.ko 2024-01-16T14:48:32.465 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge 2024-01-16T14:48:32.465 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/br_netfilter.ko 2024-01-16T14:48:32.465 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/bridge.ko 2024-01-16T14:48:32.465 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter 2024-01-16T14:48:32.465 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-01-16T14:48:32.465 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_among.ko 2024-01-16T14:48:32.465 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_arp.ko 2024-01-16T14:48:32.465 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-01-16T14:48:32.465 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-01-16T14:48:32.465 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_ip.ko 2024-01-16T14:48:32.466 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-01-16T14:48:32.466 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_limit.ko 2024-01-16T14:48:32.466 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_log.ko 2024-01-16T14:48:32.466 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_mark.ko 2024-01-16T14:48:32.466 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-01-16T14:48:32.466 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-01-16T14:48:32.466 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-01-16T14:48:32.466 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-01-16T14:48:32.466 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_snat.ko 2024-01-16T14:48:32.466 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_stp.ko 2024-01-16T14:48:32.466 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-01-16T14:48:32.466 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-01-16T14:48:32.466 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-01-16T14:48:32.466 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-01-16T14:48:32.466 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebtables.ko 2024-01-16T14:48:32.466 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-01-16T14:48:32.467 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-01-16T14:48:32.467 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-01-16T14:48:32.467 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ceph 2024-01-16T14:48:32.467 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ceph/libceph.ko 2024-01-16T14:48:32.467 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/core 2024-01-16T14:48:32.467 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/core/failover.ko 2024-01-16T14:48:32.467 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/core/pktgen.ko 2024-01-16T14:48:32.467 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dccp 2024-01-16T14:48:32.467 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dccp/dccp.ko 2024-01-16T14:48:32.467 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dccp/dccp_diag.ko 2024-01-16T14:48:32.467 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dccp/dccp_ipv4.ko 2024-01-16T14:48:32.467 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dccp/dccp_ipv6.ko 2024-01-16T14:48:32.467 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dns_resolver 2024-01-16T14:48:32.467 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dns_resolver/dns_resolver.ko 2024-01-16T14:48:32.467 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dsa 2024-01-16T14:48:32.468 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dsa/dsa_core.ko 2024-01-16T14:48:32.468 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dsa/tag_brcm.ko 2024-01-16T14:48:32.468 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dsa/tag_dsa.ko 2024-01-16T14:48:32.468 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dsa/tag_none.ko 2024-01-16T14:48:32.468 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dsa/tag_trailer.ko 2024-01-16T14:48:32.468 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ieee802154 2024-01-16T14:48:32.468 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ieee802154/6lowpan 2024-01-16T14:48:32.468 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-01-16T14:48:32.468 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ieee802154/ieee802154.ko 2024-01-16T14:48:32.468 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ieee802154/ieee802154_socket.ko 2024-01-16T14:48:32.468 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4 2024-01-16T14:48:32.468 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/ah4.ko 2024-01-16T14:48:32.468 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/esp4.ko 2024-01-16T14:48:32.468 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/fou.ko 2024-01-16T14:48:32.469 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/gre.ko 2024-01-16T14:48:32.469 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/inet_diag.ko 2024-01-16T14:48:32.469 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/ip_gre.ko 2024-01-16T14:48:32.469 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/ip_tunnel.ko 2024-01-16T14:48:32.469 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/ip_vti.ko 2024-01-16T14:48:32.469 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/ipcomp.ko 2024-01-16T14:48:32.469 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/ipip.ko 2024-01-16T14:48:32.469 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter 2024-01-16T14:48:32.469 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/arp_tables.ko 2024-01-16T14:48:32.469 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-01-16T14:48:32.469 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-01-16T14:48:32.469 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-01-16T14:48:32.469 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-01-16T14:48:32.469 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-01-16T14:48:32.469 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-01-16T14:48:32.470 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-01-16T14:48:32.470 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-01-16T14:48:32.470 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-01-16T14:48:32.470 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/iptable_security.ko 2024-01-16T14:48:32.470 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-01-16T14:48:32.470 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-01-16T14:48:32.470 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-01-16T14:48:32.470 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-01-16T14:48:32.470 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-01-16T14:48:32.470 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-01-16T14:48:32.470 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-01-16T14:48:32.470 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-01-16T14:48:32.470 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-01-16T14:48:32.470 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-01-16T14:48:32.470 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_bic.ko 2024-01-16T14:48:32.470 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_dctcp.ko 2024-01-16T14:48:32.471 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_diag.ko 2024-01-16T14:48:32.471 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_highspeed.ko 2024-01-16T14:48:32.471 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_htcp.ko 2024-01-16T14:48:32.471 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_hybla.ko 2024-01-16T14:48:32.471 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_illinois.ko 2024-01-16T14:48:32.471 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_lp.ko 2024-01-16T14:48:32.471 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_scalable.ko 2024-01-16T14:48:32.471 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_vegas.ko 2024-01-16T14:48:32.471 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_veno.ko 2024-01-16T14:48:32.471 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_westwood.ko 2024-01-16T14:48:32.471 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_yeah.ko 2024-01-16T14:48:32.471 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tunnel4.ko 2024-01-16T14:48:32.471 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/udp_diag.ko 2024-01-16T14:48:32.471 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/udp_tunnel.ko 2024-01-16T14:48:32.471 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/xfrm4_tunnel.ko 2024-01-16T14:48:32.472 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6 2024-01-16T14:48:32.472 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/ah6.ko 2024-01-16T14:48:32.472 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/esp6.ko 2024-01-16T14:48:32.472 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/fou6.ko 2024-01-16T14:48:32.472 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/ip6_tunnel.ko 2024-01-16T14:48:32.472 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-01-16T14:48:32.472 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/ip6_vti.ko 2024-01-16T14:48:32.472 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/ipcomp6.ko 2024-01-16T14:48:32.472 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter 2024-01-16T14:48:32.472 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-01-16T14:48:32.473 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-01-16T14:48:32.473 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-01-16T14:48:32.473 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-01-16T14:48:32.473 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-01-16T14:48:32.473 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-01-16T14:48:32.473 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-01-16T14:48:32.473 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-01-16T14:48:32.473 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-01-16T14:48:32.473 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-01-16T14:48:32.473 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-01-16T14:48:32.473 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-01-16T14:48:32.473 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-01-16T14:48:32.473 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-01-16T14:48:32.473 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-01-16T14:48:32.473 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-01-16T14:48:32.473 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-01-16T14:48:32.474 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-01-16T14:48:32.474 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-01-16T14:48:32.474 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-01-16T14:48:32.474 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-01-16T14:48:32.474 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-01-16T14:48:32.474 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-01-16T14:48:32.474 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-01-16T14:48:32.474 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/sit.ko 2024-01-16T14:48:32.474 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/tunnel6.ko 2024-01-16T14:48:32.474 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/xfrm6_tunnel.ko 2024-01-16T14:48:32.474 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/key 2024-01-16T14:48:32.474 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/key/af_key.ko 2024-01-16T14:48:32.474 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp 2024-01-16T14:48:32.474 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_core.ko 2024-01-16T14:48:32.474 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_debugfs.ko 2024-01-16T14:48:32.474 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_eth.ko 2024-01-16T14:48:32.475 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_ip.ko 2024-01-16T14:48:32.475 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_ip6.ko 2024-01-16T14:48:32.475 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_netlink.ko 2024-01-16T14:48:32.475 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_ppp.ko 2024-01-16T14:48:32.475 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/llc 2024-01-16T14:48:32.475 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/llc/llc.ko 2024-01-16T14:48:32.475 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/mac802154 2024-01-16T14:48:32.475 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/mac802154/mac802154.ko 2024-01-16T14:48:32.475 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/mpls 2024-01-16T14:48:32.475 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/mpls/mpls_gso.ko 2024-01-16T14:48:32.475 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter 2024-01-16T14:48:32.475 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset 2024-01-16T14:48:32.475 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set.ko 2024-01-16T14:48:32.475 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-01-16T14:48:32.475 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-01-16T14:48:32.476 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-01-16T14:48:32.476 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-01-16T14:48:32.476 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-01-16T14:48:32.476 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-01-16T14:48:32.476 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-01-16T14:48:32.476 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-01-16T14:48:32.476 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-01-16T14:48:32.476 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-01-16T14:48:32.476 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-01-16T14:48:32.476 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-01-16T14:48:32.476 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-01-16T14:48:32.476 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-01-16T14:48:32.476 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-01-16T14:48:32.476 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs 2024-01-16T14:48:32.477 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs.ko 2024-01-16T14:48:32.477 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-01-16T14:48:32.477 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-01-16T14:48:32.477 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-01-16T14:48:32.477 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-01-16T14:48:32.477 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-01-16T14:48:32.477 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-01-16T14:48:32.477 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-01-16T14:48:32.477 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-01-16T14:48:32.477 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-01-16T14:48:32.477 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-01-16T14:48:32.477 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-01-16T14:48:32.477 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-01-16T14:48:32.477 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-01-16T14:48:32.477 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conncount.ko 2024-01-16T14:48:32.477 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack.ko 2024-01-16T14:48:32.478 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-01-16T14:48:32.478 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-01-16T14:48:32.478 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-01-16T14:48:32.478 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_h323.ko 2024-01-16T14:48:32.478 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_irc.ko 2024-01-16T14:48:32.478 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-01-16T14:48:32.478 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-01-16T14:48:32.478 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-01-16T14:48:32.478 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_sane.ko 2024-01-16T14:48:32.478 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_sip.ko 2024-01-16T14:48:32.478 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-01-16T14:48:32.478 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-01-16T14:48:32.478 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_dup_netdev.ko 2024-01-16T14:48:32.478 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_flow_table.ko 2024-01-16T14:48:32.478 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_flow_table_inet.ko 2024-01-16T14:48:32.479 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_log_syslog.ko 2024-01-16T14:48:32.479 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_nat.ko 2024-01-16T14:48:32.479 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_nat_amanda.ko 2024-01-16T14:48:32.479 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_nat_ftp.ko 2024-01-16T14:48:32.479 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_nat_irc.ko 2024-01-16T14:48:32.479 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_nat_sip.ko 2024-01-16T14:48:32.479 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_nat_tftp.ko 2024-01-16T14:48:32.479 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_synproxy_core.ko 2024-01-16T14:48:32.479 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_tables.ko 2024-01-16T14:48:32.479 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nfnetlink.ko 2024-01-16T14:48:32.479 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nfnetlink_acct.ko 2024-01-16T14:48:32.479 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nfnetlink_log.ko 2024-01-16T14:48:32.479 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nfnetlink_osf.ko 2024-01-16T14:48:32.479 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nfnetlink_queue.ko 2024-01-16T14:48:32.480 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_chain_nat.ko 2024-01-16T14:48:32.480 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_compat.ko 2024-01-16T14:48:32.480 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_ct.ko 2024-01-16T14:48:32.480 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_dup_netdev.ko 2024-01-16T14:48:32.480 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_fib.ko 2024-01-16T14:48:32.480 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_fwd_netdev.ko 2024-01-16T14:48:32.480 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_hash.ko 2024-01-16T14:48:32.480 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_limit.ko 2024-01-16T14:48:32.480 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_log.ko 2024-01-16T14:48:32.480 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_masq.ko 2024-01-16T14:48:32.480 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_nat.ko 2024-01-16T14:48:32.480 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_numgen.ko 2024-01-16T14:48:32.480 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_queue.ko 2024-01-16T14:48:32.480 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_quota.ko 2024-01-16T14:48:32.481 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_redir.ko 2024-01-16T14:48:32.481 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_reject.ko 2024-01-16T14:48:32.481 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_reject_inet.ko 2024-01-16T14:48:32.481 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_socket.ko 2024-01-16T14:48:32.481 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_tunnel.ko 2024-01-16T14:48:32.481 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_AUDIT.ko 2024-01-16T14:48:32.481 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_CHECKSUM.ko 2024-01-16T14:48:32.481 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_CLASSIFY.ko 2024-01-16T14:48:32.481 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-01-16T14:48:32.481 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_CT.ko 2024-01-16T14:48:32.481 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_DSCP.ko 2024-01-16T14:48:32.481 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_HL.ko 2024-01-16T14:48:32.482 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_HMARK.ko 2024-01-16T14:48:32.482 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_IDLETIMER.ko 2024-01-16T14:48:32.482 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_LED.ko 2024-01-16T14:48:32.482 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_LOG.ko 2024-01-16T14:48:32.482 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_MASQUERADE.ko 2024-01-16T14:48:32.482 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_NETMAP.ko 2024-01-16T14:48:32.482 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_NFLOG.ko 2024-01-16T14:48:32.482 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_NFQUEUE.ko 2024-01-16T14:48:32.482 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_RATEEST.ko 2024-01-16T14:48:32.482 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_REDIRECT.ko 2024-01-16T14:48:32.482 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_SECMARK.ko 2024-01-16T14:48:32.482 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_TCPMSS.ko 2024-01-16T14:48:32.482 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-01-16T14:48:32.482 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_TEE.ko 2024-01-16T14:48:32.482 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_TPROXY.ko 2024-01-16T14:48:32.482 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_TRACE.ko 2024-01-16T14:48:32.483 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_addrtype.ko 2024-01-16T14:48:32.483 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_bpf.ko 2024-01-16T14:48:32.483 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_cgroup.ko 2024-01-16T14:48:32.483 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_cluster.ko 2024-01-16T14:48:32.483 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_comment.ko 2024-01-16T14:48:32.483 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_connbytes.ko 2024-01-16T14:48:32.483 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_connlabel.ko 2024-01-16T14:48:32.483 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_connlimit.ko 2024-01-16T14:48:32.483 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_connmark.ko 2024-01-16T14:48:32.483 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_conntrack.ko 2024-01-16T14:48:32.483 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_cpu.ko 2024-01-16T14:48:32.483 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_dccp.ko 2024-01-16T14:48:32.483 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_devgroup.ko 2024-01-16T14:48:32.483 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_dscp.ko 2024-01-16T14:48:32.483 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_ecn.ko 2024-01-16T14:48:32.483 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_esp.ko 2024-01-16T14:48:32.484 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_hashlimit.ko 2024-01-16T14:48:32.484 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_helper.ko 2024-01-16T14:48:32.484 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_hl.ko 2024-01-16T14:48:32.484 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_ipcomp.ko 2024-01-16T14:48:32.484 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_iprange.ko 2024-01-16T14:48:32.484 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_ipvs.ko 2024-01-16T14:48:32.484 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_l2tp.ko 2024-01-16T14:48:32.484 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_length.ko 2024-01-16T14:48:32.484 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_limit.ko 2024-01-16T14:48:32.484 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_mac.ko 2024-01-16T14:48:32.484 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_mark.ko 2024-01-16T14:48:32.484 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_multiport.ko 2024-01-16T14:48:32.484 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_nat.ko 2024-01-16T14:48:32.484 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_nfacct.ko 2024-01-16T14:48:32.485 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_osf.ko 2024-01-16T14:48:32.485 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_owner.ko 2024-01-16T14:48:32.485 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_physdev.ko 2024-01-16T14:48:32.485 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_pkttype.ko 2024-01-16T14:48:32.485 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_policy.ko 2024-01-16T14:48:32.485 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_quota.ko 2024-01-16T14:48:32.485 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_rateest.ko 2024-01-16T14:48:32.485 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_realm.ko 2024-01-16T14:48:32.485 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_recent.ko 2024-01-16T14:48:32.485 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_sctp.ko 2024-01-16T14:48:32.485 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_set.ko 2024-01-16T14:48:32.485 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_socket.ko 2024-01-16T14:48:32.485 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_state.ko 2024-01-16T14:48:32.485 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_statistic.ko 2024-01-16T14:48:32.485 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_string.ko 2024-01-16T14:48:32.485 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_tcpmss.ko 2024-01-16T14:48:32.486 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_time.ko 2024-01-16T14:48:32.486 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_u32.ko 2024-01-16T14:48:32.486 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netlink 2024-01-16T14:48:32.486 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netlink/netlink_diag.ko 2024-01-16T14:48:32.486 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/nsh 2024-01-16T14:48:32.486 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/nsh/nsh.ko 2024-01-16T14:48:32.486 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/openvswitch 2024-01-16T14:48:32.486 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/openvswitch/openvswitch.ko 2024-01-16T14:48:32.486 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/openvswitch/vport-geneve.ko 2024-01-16T14:48:32.486 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/openvswitch/vport-gre.ko 2024-01-16T14:48:32.486 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/openvswitch/vport-vxlan.ko 2024-01-16T14:48:32.486 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/packet 2024-01-16T14:48:32.486 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/packet/af_packet_diag.ko 2024-01-16T14:48:32.486 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/rds 2024-01-16T14:48:32.486 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/rds/rds.ko 2024-01-16T14:48:32.487 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/rds/rds_rdma.ko 2024-01-16T14:48:32.487 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/rds/rds_tcp.ko 2024-01-16T14:48:32.487 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched 2024-01-16T14:48:32.487 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_csum.ko 2024-01-16T14:48:32.487 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_gact.ko 2024-01-16T14:48:32.487 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_ipt.ko 2024-01-16T14:48:32.487 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_mirred.ko 2024-01-16T14:48:32.487 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_nat.ko 2024-01-16T14:48:32.487 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_pedit.ko 2024-01-16T14:48:32.487 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_police.ko 2024-01-16T14:48:32.487 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_simple.ko 2024-01-16T14:48:32.487 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_skbedit.ko 2024-01-16T14:48:32.487 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_vlan.ko 2024-01-16T14:48:32.488 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/cls_basic.ko 2024-01-16T14:48:32.488 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/cls_bpf.ko 2024-01-16T14:48:32.488 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/cls_flow.ko 2024-01-16T14:48:32.488 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/cls_fw.ko 2024-01-16T14:48:32.488 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/cls_route.ko 2024-01-16T14:48:32.488 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/cls_u32.ko 2024-01-16T14:48:32.488 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/em_cmp.ko 2024-01-16T14:48:32.488 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/em_ipset.ko 2024-01-16T14:48:32.488 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/em_meta.ko 2024-01-16T14:48:32.488 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/em_nbyte.ko 2024-01-16T14:48:32.488 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/em_text.ko 2024-01-16T14:48:32.488 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/em_u32.ko 2024-01-16T14:48:32.488 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_choke.ko 2024-01-16T14:48:32.488 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_codel.ko 2024-01-16T14:48:32.489 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_drr.ko 2024-01-16T14:48:32.489 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_fq.ko 2024-01-16T14:48:32.489 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_gred.ko 2024-01-16T14:48:32.489 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_hfsc.ko 2024-01-16T14:48:32.489 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_hhf.ko 2024-01-16T14:48:32.489 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_htb.ko 2024-01-16T14:48:32.489 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_ingress.ko 2024-01-16T14:48:32.489 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_mqprio.ko 2024-01-16T14:48:32.489 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_mqprio_lib.ko 2024-01-16T14:48:32.489 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_multiq.ko 2024-01-16T14:48:32.489 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_netem.ko 2024-01-16T14:48:32.489 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_pie.ko 2024-01-16T14:48:32.489 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_plug.ko 2024-01-16T14:48:32.489 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_prio.ko 2024-01-16T14:48:32.490 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_qfq.ko 2024-01-16T14:48:32.490 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_red.ko 2024-01-16T14:48:32.490 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_sfb.ko 2024-01-16T14:48:32.490 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_sfq.ko 2024-01-16T14:48:32.490 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_tbf.ko 2024-01-16T14:48:32.490 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_teql.ko 2024-01-16T14:48:32.490 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sctp 2024-01-16T14:48:32.490 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sctp/sctp.ko 2024-01-16T14:48:32.490 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sctp/sctp_diag.ko 2024-01-16T14:48:32.490 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc 2024-01-16T14:48:32.490 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc/auth_gss 2024-01-16T14:48:32.490 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-01-16T14:48:32.490 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-01-16T14:48:32.490 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc/sunrpc.ko 2024-01-16T14:48:32.490 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc/xprtrdma 2024-01-16T14:48:32.490 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-01-16T14:48:32.491 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/tipc 2024-01-16T14:48:32.491 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/tipc/diag.ko 2024-01-16T14:48:32.491 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/tipc/tipc.ko 2024-01-16T14:48:32.491 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/unix 2024-01-16T14:48:32.491 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/unix/unix_diag.ko 2024-01-16T14:48:32.491 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/vmw_vsock 2024-01-16T14:48:32.491 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-01-16T14:48:32.491 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-01-16T14:48:32.491 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/vmw_vsock/vsock.ko 2024-01-16T14:48:32.491 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/vmw_vsock/vsock_diag.ko 2024-01-16T14:48:32.491 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/vmw_vsock/vsock_loopback.ko 2024-01-16T14:48:32.491 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/xfrm 2024-01-16T14:48:32.491 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/xfrm/xfrm_ipcomp.ko 2024-01-16T14:48:32.491 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/security 2024-01-16T14:48:32.491 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/security/keys 2024-01-16T14:48:32.491 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/security/keys/encrypted-keys 2024-01-16T14:48:32.492 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-01-16T14:48:32.492 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/security/keys/trusted-keys 2024-01-16T14:48:32.492 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/security/keys/trusted-keys/trusted.ko 2024-01-16T14:48:32.492 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/virt 2024-01-16T14:48:32.492 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/virt/lib 2024-01-16T14:48:32.492 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/virt/lib/irqbypass.ko 2024-01-16T14:48:32.492 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/modules.alias 2024-01-16T14:48:32.492 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/modules.alias.bin 2024-01-16T14:48:32.492 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/modules.builtin 2024-01-16T14:48:32.492 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/modules.builtin.alias.bin 2024-01-16T14:48:32.492 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/modules.builtin.bin 2024-01-16T14:48:32.492 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/modules.builtin.modinfo 2024-01-16T14:48:32.492 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/modules.dep 2024-01-16T14:48:32.493 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/modules.dep.bin 2024-01-16T14:48:32.493 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/modules.devname 2024-01-16T14:48:32.493 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/modules.order 2024-01-16T14:48:32.493 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/modules.softdep 2024-01-16T14:48:32.493 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/modules.symbols 2024-01-16T14:48:32.493 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.7.0-g2668949e0971/modules.symbols.bin 2024-01-16T14:48:32.493 DEBUG:teuthology.task.kernel:get_image_version: 6.7.0-g2668949e0971 2024-01-16T14:48:32.494 DEBUG:teuthology.orchestra.run.smithi143:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-01-16T14:48:32.515 INFO:teuthology.orchestra.run.smithi143.stdout:postinstall scriptlet (using /bin/sh): 2024-01-16T14:48:32.515 INFO:teuthology.orchestra.run.smithi143.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.7.0-g2668949e0971 -a -r /boot/System.map-6.7.0-g2668949e0971 ]; then 2024-01-16T14:48:32.515 INFO:teuthology.orchestra.run.smithi143.stdout:cp /boot/vmlinuz-6.7.0-g2668949e0971 /boot/.vmlinuz-6.7.0-g2668949e0971-rpm 2024-01-16T14:48:32.515 INFO:teuthology.orchestra.run.smithi143.stdout:cp /boot/System.map-6.7.0-g2668949e0971 /boot/.System.map-6.7.0-g2668949e0971-rpm 2024-01-16T14:48:32.516 INFO:teuthology.orchestra.run.smithi143.stdout:rm -f /boot/vmlinuz-6.7.0-g2668949e0971 /boot/System.map-6.7.0-g2668949e0971 2024-01-16T14:48:32.516 INFO:teuthology.orchestra.run.smithi143.stdout:/sbin/installkernel 6.7.0-g2668949e0971 /boot/.vmlinuz-6.7.0-g2668949e0971-rpm /boot/.System.map-6.7.0-g2668949e0971-rpm 2024-01-16T14:48:32.516 INFO:teuthology.orchestra.run.smithi143.stdout:rm -f /boot/.vmlinuz-6.7.0-g2668949e0971-rpm /boot/.System.map-6.7.0-g2668949e0971-rpm 2024-01-16T14:48:32.516 INFO:teuthology.orchestra.run.smithi143.stdout:fi 2024-01-16T14:48:32.516 INFO:teuthology.orchestra.run.smithi143.stdout:preuninstall scriptlet (using /bin/sh): 2024-01-16T14:48:32.516 INFO:teuthology.orchestra.run.smithi143.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-01-16T14:48:32.516 INFO:teuthology.orchestra.run.smithi143.stdout:new-kernel-pkg --remove 6.7.0-g2668949e0971 --rminitrd --initrdfile=/boot/initramfs-6.7.0-g2668949e0971.img 2024-01-16T14:48:32.516 INFO:teuthology.orchestra.run.smithi143.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-01-16T14:48:32.516 INFO:teuthology.orchestra.run.smithi143.stdout:kernel-install remove 6.7.0-g2668949e0971 2024-01-16T14:48:32.516 INFO:teuthology.orchestra.run.smithi143.stdout:fi 2024-01-16T14:48:32.516 INFO:teuthology.orchestra.run.smithi143.stdout:postuninstall scriptlet (using /bin/sh): 2024-01-16T14:48:32.516 INFO:teuthology.orchestra.run.smithi143.stdout:if [ -x /sbin/update-bootloader ]; then 2024-01-16T14:48:32.516 INFO:teuthology.orchestra.run.smithi143.stdout:/sbin/update-bootloader --remove 6.7.0-g2668949e0971 2024-01-16T14:48:32.516 INFO:teuthology.orchestra.run.smithi143.stdout:fi 2024-01-16T14:48:32.517 DEBUG:teuthology.orchestra.run.smithi143:> sudo rpm -qi grub2-tools 2024-01-16T14:48:32.592 INFO:teuthology.orchestra.run.smithi143.stdout:Name : grub2-tools 2024-01-16T14:48:32.592 INFO:teuthology.orchestra.run.smithi143.stdout:Epoch : 1 2024-01-16T14:48:32.592 INFO:teuthology.orchestra.run.smithi143.stdout:Version : 2.06 2024-01-16T14:48:32.592 INFO:teuthology.orchestra.run.smithi143.stdout:Release : 68.el9 2024-01-16T14:48:32.592 INFO:teuthology.orchestra.run.smithi143.stdout:Architecture: x86_64 2024-01-16T14:48:32.593 INFO:teuthology.orchestra.run.smithi143.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-01-16T14:48:32.593 INFO:teuthology.orchestra.run.smithi143.stdout:Group : Unspecified 2024-01-16T14:48:32.593 INFO:teuthology.orchestra.run.smithi143.stdout:Size : 8264254 2024-01-16T14:48:32.593 INFO:teuthology.orchestra.run.smithi143.stdout:License : GPLv3+ 2024-01-16T14:48:32.593 INFO:teuthology.orchestra.run.smithi143.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-01-16T14:48:32.593 INFO:teuthology.orchestra.run.smithi143.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-01-16T14:48:32.593 INFO:teuthology.orchestra.run.smithi143.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-01-16T14:48:32.593 INFO:teuthology.orchestra.run.smithi143.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-01-16T14:48:32.593 INFO:teuthology.orchestra.run.smithi143.stdout:Packager : builder@centos.org 2024-01-16T14:48:32.593 INFO:teuthology.orchestra.run.smithi143.stdout:Vendor : CentOS 2024-01-16T14:48:32.593 INFO:teuthology.orchestra.run.smithi143.stdout:URL : http://www.gnu.org/software/grub/ 2024-01-16T14:48:32.593 INFO:teuthology.orchestra.run.smithi143.stdout:Summary : Support tools for GRUB. 2024-01-16T14:48:32.593 INFO:teuthology.orchestra.run.smithi143.stdout:Description : 2024-01-16T14:48:32.593 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:48:32.593 INFO:teuthology.orchestra.run.smithi143.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-01-16T14:48:32.593 INFO:teuthology.orchestra.run.smithi143.stdout:customizable bootloader with modular architecture. It supports a rich 2024-01-16T14:48:32.594 INFO:teuthology.orchestra.run.smithi143.stdout:variety of kernel formats, file systems, computer architectures and 2024-01-16T14:48:32.594 INFO:teuthology.orchestra.run.smithi143.stdout:hardware devices. 2024-01-16T14:48:32.594 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:48:32.594 INFO:teuthology.orchestra.run.smithi143.stdout:This subpackage provides tools for support of all platforms. 2024-01-16T14:48:32.594 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-01-16T14:48:32.594 INFO:teuthology.task.kernel:Updating grub on smithi143 to boot 6.7.0-g2668949e0971 2024-01-16T14:48:32.595 DEBUG:teuthology.orchestra.run.smithi143:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-01-16T14:48:33.233 INFO:teuthology.orchestra.run.smithi143.stderr:Generating grub configuration file ... 2024-01-16T14:48:34.651 INFO:teuthology.orchestra.run.smithi143.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-01-16T14:48:34.684 INFO:teuthology.orchestra.run.smithi143.stderr:done 2024-01-16T14:48:34.687 DEBUG:teuthology.orchestra.run.smithi143:> mktemp 2024-01-16T14:48:34.701 INFO:teuthology.orchestra.run.smithi143.stdout:/tmp/tmp.WqfimgBgCV 2024-01-16T14:48:34.701 DEBUG:teuthology.orchestra.run.smithi143:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.WqfimgBgCV 2024-01-16T14:48:34.769 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 0666 /tmp/tmp.WqfimgBgCV 2024-01-16T14:48:34.908 DEBUG:teuthology.orchestra.remote:smithi143:/tmp/tmp.WqfimgBgCV is 6KB 2024-01-16T14:48:34.957 DEBUG:teuthology.orchestra.run.smithi143:> rm -fr /tmp/tmp.WqfimgBgCV 2024-01-16T14:48:34.971 DEBUG:teuthology.orchestra.run.smithi143:> sudo /bin/ls /boot/loader/entries || true 2024-01-16T14:48:35.036 INFO:teuthology.orchestra.run.smithi143.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-01-16T14:48:35.037 INFO:teuthology.orchestra.run.smithi143.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-01-16T14:48:35.037 INFO:teuthology.orchestra.run.smithi143.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-01-16T14:48:35.037 INFO:teuthology.orchestra.run.smithi143.stdout:15949e3c88704300882482eddfd7baae-6.7.0-g2668949e0971.conf 2024-01-16T14:48:35.038 DEBUG:teuthology.orchestra.run.smithi143:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.7.0-g2668949e0971 2024-01-16T14:48:35.178 DEBUG:teuthology.orchestra.run.smithi143:> sudo shutdown -r now 2024-01-16T14:48:50.432 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi204.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-01-16T14:48:50.432 DEBUG:teuthology.orchestra.run.smithi204:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-01-16T14:48:50.511 INFO:teuthology.orchestra.run.smithi204.stdout:/boot/System.map-6.7.0-g2668949e0971 2024-01-16T14:48:50.511 INFO:teuthology.orchestra.run.smithi204.stdout:/boot/config-6.7.0-g2668949e0971 2024-01-16T14:48:50.511 INFO:teuthology.orchestra.run.smithi204.stdout:/boot/vmlinuz-6.7.0-g2668949e0971 2024-01-16T14:48:50.511 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971 2024-01-16T14:48:50.511 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel 2024-01-16T14:48:50.511 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch 2024-01-16T14:48:50.511 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86 2024-01-16T14:48:50.511 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto 2024-01-16T14:48:50.511 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-01-16T14:48:50.512 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-01-16T14:48:50.512 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-01-16T14:48:50.512 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-01-16T14:48:50.512 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-01-16T14:48:50.512 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-01-16T14:48:50.512 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-01-16T14:48:50.512 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/crc32c-intel.ko 2024-01-16T14:48:50.512 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-01-16T14:48:50.513 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-01-16T14:48:50.513 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-01-16T14:48:50.513 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-01-16T14:48:50.513 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/serpent-avx2.ko 2024-01-16T14:48:50.513 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-01-16T14:48:50.513 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-01-16T14:48:50.513 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-01-16T14:48:50.513 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-01-16T14:48:50.513 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-01-16T14:48:50.513 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-01-16T14:48:50.513 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-01-16T14:48:50.513 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kernel 2024-01-16T14:48:50.513 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kernel/cpu 2024-01-16T14:48:50.514 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kernel/cpu/mce 2024-01-16T14:48:50.514 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-01-16T14:48:50.514 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kvm 2024-01-16T14:48:50.514 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kvm/kvm-amd.ko 2024-01-16T14:48:50.514 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kvm/kvm-intel.ko 2024-01-16T14:48:50.514 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kvm/kvm.ko 2024-01-16T14:48:50.514 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto 2024-01-16T14:48:50.514 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/ansi_cprng.ko 2024-01-16T14:48:50.514 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/anubis.ko 2024-01-16T14:48:50.514 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/arc4.ko 2024-01-16T14:48:50.514 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx 2024-01-16T14:48:50.514 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx/async_memcpy.ko 2024-01-16T14:48:50.514 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx/async_pq.ko 2024-01-16T14:48:50.514 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx/async_raid6_recov.ko 2024-01-16T14:48:50.515 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx/async_tx.ko 2024-01-16T14:48:50.515 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx/async_xor.ko 2024-01-16T14:48:50.515 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx/raid6test.ko 2024-01-16T14:48:50.515 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/authenc.ko 2024-01-16T14:48:50.515 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/authencesn.ko 2024-01-16T14:48:50.515 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/blake2b_generic.ko 2024-01-16T14:48:50.515 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/blowfish_common.ko 2024-01-16T14:48:50.515 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/blowfish_generic.ko 2024-01-16T14:48:50.515 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/camellia_generic.ko 2024-01-16T14:48:50.515 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/cast5_generic.ko 2024-01-16T14:48:50.515 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/cast6_generic.ko 2024-01-16T14:48:50.515 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/cast_common.ko 2024-01-16T14:48:50.515 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/ccm.ko 2024-01-16T14:48:50.515 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/cmac.ko 2024-01-16T14:48:50.515 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/crc32_generic.ko 2024-01-16T14:48:50.516 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/crypto_engine.ko 2024-01-16T14:48:50.516 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/crypto_user.ko 2024-01-16T14:48:50.516 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/cts.ko 2024-01-16T14:48:50.516 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/des_generic.ko 2024-01-16T14:48:50.516 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/dh_generic.ko 2024-01-16T14:48:50.516 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/echainiv.ko 2024-01-16T14:48:50.516 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/essiv.ko 2024-01-16T14:48:50.516 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/fcrypt.ko 2024-01-16T14:48:50.516 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/khazad.ko 2024-01-16T14:48:50.516 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/lz4.ko 2024-01-16T14:48:50.516 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/lz4hc.ko 2024-01-16T14:48:50.516 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/md4.ko 2024-01-16T14:48:50.516 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/michael_mic.ko 2024-01-16T14:48:50.516 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/pcbc.ko 2024-01-16T14:48:50.517 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/pcrypt.ko 2024-01-16T14:48:50.517 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/rmd160.ko 2024-01-16T14:48:50.517 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/seed.ko 2024-01-16T14:48:50.517 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/serpent_generic.ko 2024-01-16T14:48:50.517 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/tcrypt.ko 2024-01-16T14:48:50.517 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/tea.ko 2024-01-16T14:48:50.517 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/twofish_common.ko 2024-01-16T14:48:50.517 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/twofish_generic.ko 2024-01-16T14:48:50.517 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/vmac.ko 2024-01-16T14:48:50.517 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/wp512.ko 2024-01-16T14:48:50.517 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/xcbc.ko 2024-01-16T14:48:50.517 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/xor.ko 2024-01-16T14:48:50.517 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/xxhash_generic.ko 2024-01-16T14:48:50.517 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers 2024-01-16T14:48:50.518 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi 2024-01-16T14:48:50.518 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/acpi_ipmi.ko 2024-01-16T14:48:50.518 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/acpi_pad.ko 2024-01-16T14:48:50.518 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/custom_method.ko 2024-01-16T14:48:50.518 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/ec_sys.ko 2024-01-16T14:48:50.518 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/platform_profile.ko 2024-01-16T14:48:50.518 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/sbs.ko 2024-01-16T14:48:50.518 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/sbshc.ko 2024-01-16T14:48:50.518 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/video.ko 2024-01-16T14:48:50.518 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata 2024-01-16T14:48:50.518 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/acard-ahci.ko 2024-01-16T14:48:50.518 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/ahci_platform.ko 2024-01-16T14:48:50.518 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/ata_generic.ko 2024-01-16T14:48:50.518 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/libahci_platform.ko 2024-01-16T14:48:50.518 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_acpi.ko 2024-01-16T14:48:50.518 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_ali.ko 2024-01-16T14:48:50.519 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_amd.ko 2024-01-16T14:48:50.519 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_artop.ko 2024-01-16T14:48:50.519 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_atiixp.ko 2024-01-16T14:48:50.519 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_atp867x.ko 2024-01-16T14:48:50.519 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_cmd640.ko 2024-01-16T14:48:50.519 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_cmd64x.ko 2024-01-16T14:48:50.519 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_cypress.ko 2024-01-16T14:48:50.519 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_efar.ko 2024-01-16T14:48:50.519 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_hpt366.ko 2024-01-16T14:48:50.519 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_hpt37x.ko 2024-01-16T14:48:50.519 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_hpt3x2n.ko 2024-01-16T14:48:50.519 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_hpt3x3.ko 2024-01-16T14:48:50.519 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_it8213.ko 2024-01-16T14:48:50.519 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_it821x.ko 2024-01-16T14:48:50.519 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_jmicron.ko 2024-01-16T14:48:50.520 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_marvell.ko 2024-01-16T14:48:50.520 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_mpiix.ko 2024-01-16T14:48:50.520 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_netcell.ko 2024-01-16T14:48:50.520 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_ninja32.ko 2024-01-16T14:48:50.520 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_ns87410.ko 2024-01-16T14:48:50.520 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_ns87415.ko 2024-01-16T14:48:50.520 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_oldpiix.ko 2024-01-16T14:48:50.520 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_opti.ko 2024-01-16T14:48:50.520 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_optidma.ko 2024-01-16T14:48:50.520 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_pcmcia.ko 2024-01-16T14:48:50.520 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_pdc2027x.ko 2024-01-16T14:48:50.520 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-01-16T14:48:50.520 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_piccolo.ko 2024-01-16T14:48:50.520 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_rdc.ko 2024-01-16T14:48:50.521 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_sch.ko 2024-01-16T14:48:50.521 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_serverworks.ko 2024-01-16T14:48:50.521 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_sil680.ko 2024-01-16T14:48:50.521 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_sis.ko 2024-01-16T14:48:50.521 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_sl82c105.ko 2024-01-16T14:48:50.521 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_triflex.ko 2024-01-16T14:48:50.521 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_via.ko 2024-01-16T14:48:50.521 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pdc_adma.ko 2024-01-16T14:48:50.521 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_inic162x.ko 2024-01-16T14:48:50.521 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_mv.ko 2024-01-16T14:48:50.521 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_nv.ko 2024-01-16T14:48:50.521 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_promise.ko 2024-01-16T14:48:50.521 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_qstor.ko 2024-01-16T14:48:50.521 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_sil.ko 2024-01-16T14:48:50.521 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_sil24.ko 2024-01-16T14:48:50.521 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_sis.ko 2024-01-16T14:48:50.522 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_svw.ko 2024-01-16T14:48:50.522 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_sx4.ko 2024-01-16T14:48:50.522 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_uli.ko 2024-01-16T14:48:50.522 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_via.ko 2024-01-16T14:48:50.522 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_vsc.ko 2024-01-16T14:48:50.522 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/auxdisplay 2024-01-16T14:48:50.522 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/auxdisplay/ks0108.ko 2024-01-16T14:48:50.522 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/base 2024-01-16T14:48:50.522 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/base/regmap 2024-01-16T14:48:50.522 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/base/regmap/regmap-i2c.ko 2024-01-16T14:48:50.522 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/base/regmap/regmap-mmio.ko 2024-01-16T14:48:50.522 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/bcma 2024-01-16T14:48:50.522 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/bcma/bcma.ko 2024-01-16T14:48:50.522 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block 2024-01-16T14:48:50.522 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/aoe 2024-01-16T14:48:50.522 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/aoe/aoe.ko 2024-01-16T14:48:50.523 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/brd.ko 2024-01-16T14:48:50.523 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/drbd 2024-01-16T14:48:50.523 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/drbd/drbd.ko 2024-01-16T14:48:50.523 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/floppy.ko 2024-01-16T14:48:50.523 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/loop.ko 2024-01-16T14:48:50.523 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/mtip32xx 2024-01-16T14:48:50.523 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-01-16T14:48:50.523 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/nbd.ko 2024-01-16T14:48:50.523 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/null_blk 2024-01-16T14:48:50.523 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/null_blk/null_blk.ko 2024-01-16T14:48:50.523 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/pktcdvd.ko 2024-01-16T14:48:50.523 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/rbd.ko 2024-01-16T14:48:50.523 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/virtio_blk.ko 2024-01-16T14:48:50.523 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/xen-blkback 2024-01-16T14:48:50.523 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-01-16T14:48:50.523 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/xen-blkfront.ko 2024-01-16T14:48:50.524 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/zram 2024-01-16T14:48:50.524 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/zram/zram.ko 2024-01-16T14:48:50.524 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char 2024-01-16T14:48:50.524 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hangcheck-timer.ko 2024-01-16T14:48:50.524 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hw_random 2024-01-16T14:48:50.524 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hw_random/amd-rng.ko 2024-01-16T14:48:50.524 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hw_random/intel-rng.ko 2024-01-16T14:48:50.524 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-01-16T14:48:50.524 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hw_random/via-rng.ko 2024-01-16T14:48:50.524 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hw_random/virtio-rng.ko 2024-01-16T14:48:50.524 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi 2024-01-16T14:48:50.524 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-01-16T14:48:50.524 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-01-16T14:48:50.524 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-01-16T14:48:50.525 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi/ipmi_si.ko 2024-01-16T14:48:50.525 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-01-16T14:48:50.525 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-01-16T14:48:50.525 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/lp.ko 2024-01-16T14:48:50.525 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/mwave 2024-01-16T14:48:50.525 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/mwave/mwave.ko 2024-01-16T14:48:50.525 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ppdev.ko 2024-01-16T14:48:50.525 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tlclk.ko 2024-01-16T14:48:50.525 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm 2024-01-16T14:48:50.525 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/tpm.ko 2024-01-16T14:48:50.525 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/tpm_atmel.ko 2024-01-16T14:48:50.525 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/tpm_infineon.ko 2024-01-16T14:48:50.525 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/tpm_nsc.ko 2024-01-16T14:48:50.525 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/tpm_tis.ko 2024-01-16T14:48:50.525 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-01-16T14:48:50.526 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/uv_mmtimer.ko 2024-01-16T14:48:50.526 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/virtio_console.ko 2024-01-16T14:48:50.526 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/cpufreq 2024-01-16T14:48:50.526 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-01-16T14:48:50.526 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-01-16T14:48:50.526 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/cpufreq/p4-clockmod.ko 2024-01-16T14:48:50.526 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-01-16T14:48:50.526 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/cpufreq/powernow-k8.ko 2024-01-16T14:48:50.526 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/cpufreq/speedstep-lib.ko 2024-01-16T14:48:50.526 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto 2024-01-16T14:48:50.526 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/ccp 2024-01-16T14:48:50.526 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-01-16T14:48:50.526 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/ccp/ccp.ko 2024-01-16T14:48:50.526 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/intel 2024-01-16T14:48:50.526 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/intel/qat 2024-01-16T14:48:50.526 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/intel/qat/qat_common 2024-01-16T14:48:50.526 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-01-16T14:48:50.527 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-01-16T14:48:50.527 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-01-16T14:48:50.527 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/padlock-aes.ko 2024-01-16T14:48:50.527 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/padlock-sha.ko 2024-01-16T14:48:50.527 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/virtio 2024-01-16T14:48:50.527 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-01-16T14:48:50.527 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/dca 2024-01-16T14:48:50.527 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/dca/dca.ko 2024-01-16T14:48:50.527 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/devfreq 2024-01-16T14:48:50.527 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-01-16T14:48:50.527 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/dma 2024-01-16T14:48:50.527 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/dma/dw 2024-01-16T14:48:50.527 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/dma/dw/dw_dmac.ko 2024-01-16T14:48:50.527 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/dma/ioat 2024-01-16T14:48:50.527 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/dma/ioat/ioatdma.ko 2024-01-16T14:48:50.528 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac 2024-01-16T14:48:50.528 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/amd64_edac.ko 2024-01-16T14:48:50.528 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/e752x_edac.ko 2024-01-16T14:48:50.528 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/edac_mce_amd.ko 2024-01-16T14:48:50.528 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i3000_edac.ko 2024-01-16T14:48:50.528 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i3200_edac.ko 2024-01-16T14:48:50.528 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i5100_edac.ko 2024-01-16T14:48:50.528 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i5400_edac.ko 2024-01-16T14:48:50.528 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i7300_edac.ko 2024-01-16T14:48:50.528 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i7core_edac.ko 2024-01-16T14:48:50.528 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i82975x_edac.ko 2024-01-16T14:48:50.528 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/ie31200_edac.ko 2024-01-16T14:48:50.528 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/sb_edac.ko 2024-01-16T14:48:50.528 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/x38_edac.ko 2024-01-16T14:48:50.528 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/firmware 2024-01-16T14:48:50.529 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/firmware/edd.ko 2024-01-16T14:48:50.529 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/firmware/iscsi_ibft.ko 2024-01-16T14:48:50.529 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio 2024-01-16T14:48:50.529 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-ich.ko 2024-01-16T14:48:50.529 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-viperboard.ko 2024-01-16T14:48:50.529 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid 2024-01-16T14:48:50.529 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-appleir.ko 2024-01-16T14:48:50.529 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-aureal.ko 2024-01-16T14:48:50.529 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-axff.ko 2024-01-16T14:48:50.529 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-dr.ko 2024-01-16T14:48:50.529 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-elecom.ko 2024-01-16T14:48:50.529 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-elo.ko 2024-01-16T14:48:50.529 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-emsff.ko 2024-01-16T14:48:50.529 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-gaff.ko 2024-01-16T14:48:50.529 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-gt683r.ko 2024-01-16T14:48:50.529 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-gyration.ko 2024-01-16T14:48:50.530 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-holtek-kbd.ko 2024-01-16T14:48:50.530 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-holtek-mouse.ko 2024-01-16T14:48:50.530 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-holtekff.ko 2024-01-16T14:48:50.530 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-hyperv.ko 2024-01-16T14:48:50.530 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-icade.ko 2024-01-16T14:48:50.530 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-keytouch.ko 2024-01-16T14:48:50.530 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-kye.ko 2024-01-16T14:48:50.530 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-lcpower.ko 2024-01-16T14:48:50.530 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-led.ko 2024-01-16T14:48:50.530 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-lenovo.ko 2024-01-16T14:48:50.530 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-logitech-dj.ko 2024-01-16T14:48:50.530 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-01-16T14:48:50.530 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-multitouch.ko 2024-01-16T14:48:50.530 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-ortek.ko 2024-01-16T14:48:50.531 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-penmount.ko 2024-01-16T14:48:50.531 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-petalynx.ko 2024-01-16T14:48:50.531 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-picolcd.ko 2024-01-16T14:48:50.531 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-pl.ko 2024-01-16T14:48:50.531 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-plantronics.ko 2024-01-16T14:48:50.531 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-primax.ko 2024-01-16T14:48:50.531 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-rmi.ko 2024-01-16T14:48:50.531 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-arvo.ko 2024-01-16T14:48:50.531 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-common.ko 2024-01-16T14:48:50.531 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-isku.ko 2024-01-16T14:48:50.531 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-kone.ko 2024-01-16T14:48:50.531 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-01-16T14:48:50.531 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-konepure.ko 2024-01-16T14:48:50.531 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-01-16T14:48:50.531 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-lua.ko 2024-01-16T14:48:50.531 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-pyra.ko 2024-01-16T14:48:50.532 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-ryos.ko 2024-01-16T14:48:50.532 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-savu.ko 2024-01-16T14:48:50.532 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat.ko 2024-01-16T14:48:50.532 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-saitek.ko 2024-01-16T14:48:50.532 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-samsung.ko 2024-01-16T14:48:50.532 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-sensor-hub.ko 2024-01-16T14:48:50.532 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-sjoy.ko 2024-01-16T14:48:50.532 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-sony.ko 2024-01-16T14:48:50.532 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-speedlink.ko 2024-01-16T14:48:50.532 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-steelseries.ko 2024-01-16T14:48:50.532 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-sunplus.ko 2024-01-16T14:48:50.532 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-thrustmaster.ko 2024-01-16T14:48:50.532 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-tivo.ko 2024-01-16T14:48:50.532 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-tmff.ko 2024-01-16T14:48:50.532 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-topseed.ko 2024-01-16T14:48:50.532 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-twinhan.ko 2024-01-16T14:48:50.533 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-uclogic.ko 2024-01-16T14:48:50.533 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-waltop.ko 2024-01-16T14:48:50.533 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-wiimote.ko 2024-01-16T14:48:50.533 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-xinmo.ko 2024-01-16T14:48:50.533 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-zpff.ko 2024-01-16T14:48:50.533 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-zydacron.ko 2024-01-16T14:48:50.533 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/uhid.ko 2024-01-16T14:48:50.533 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/wacom.ko 2024-01-16T14:48:50.533 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hv 2024-01-16T14:48:50.533 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hv/hv_balloon.ko 2024-01-16T14:48:50.533 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hv/hv_utils.ko 2024-01-16T14:48:50.533 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hv/hv_vmbus.ko 2024-01-16T14:48:50.533 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon 2024-01-16T14:48:50.533 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/abituguru.ko 2024-01-16T14:48:50.533 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/abituguru3.ko 2024-01-16T14:48:50.533 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/acpi_power_meter.ko 2024-01-16T14:48:50.534 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ad7414.ko 2024-01-16T14:48:50.534 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ad7418.ko 2024-01-16T14:48:50.534 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adc128d818.ko 2024-01-16T14:48:50.534 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adm1025.ko 2024-01-16T14:48:50.534 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adm1026.ko 2024-01-16T14:48:50.534 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adm1029.ko 2024-01-16T14:48:50.534 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adm1031.ko 2024-01-16T14:48:50.534 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adm9240.ko 2024-01-16T14:48:50.534 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ads7828.ko 2024-01-16T14:48:50.534 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adt7410.ko 2024-01-16T14:48:50.534 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adt7411.ko 2024-01-16T14:48:50.534 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adt7462.ko 2024-01-16T14:48:50.534 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adt7470.ko 2024-01-16T14:48:50.534 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adt7475.ko 2024-01-16T14:48:50.534 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adt7x10.ko 2024-01-16T14:48:50.534 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/amc6821.ko 2024-01-16T14:48:50.535 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/applesmc.ko 2024-01-16T14:48:50.535 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/asb100.ko 2024-01-16T14:48:50.535 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/asc7621.ko 2024-01-16T14:48:50.535 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/asus_atk0110.ko 2024-01-16T14:48:50.535 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/atxp1.ko 2024-01-16T14:48:50.535 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/coretemp.ko 2024-01-16T14:48:50.535 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-01-16T14:48:50.535 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/dme1737.ko 2024-01-16T14:48:50.535 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ds1621.ko 2024-01-16T14:48:50.535 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ds620.ko 2024-01-16T14:48:50.535 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/emc1403.ko 2024-01-16T14:48:50.535 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/emc6w201.ko 2024-01-16T14:48:50.535 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/f71805f.ko 2024-01-16T14:48:50.535 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/f71882fg.ko 2024-01-16T14:48:50.535 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/f75375s.ko 2024-01-16T14:48:50.536 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/fam15h_power.ko 2024-01-16T14:48:50.536 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/fschmd.ko 2024-01-16T14:48:50.536 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/g760a.ko 2024-01-16T14:48:50.536 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/g762.ko 2024-01-16T14:48:50.536 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/gl518sm.ko 2024-01-16T14:48:50.536 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/gl520sm.ko 2024-01-16T14:48:50.536 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/hwmon-vid.ko 2024-01-16T14:48:50.536 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/i5500_temp.ko 2024-01-16T14:48:50.536 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/i5k_amb.ko 2024-01-16T14:48:50.536 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ibmaem.ko 2024-01-16T14:48:50.536 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ibmpex.ko 2024-01-16T14:48:50.536 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ina209.ko 2024-01-16T14:48:50.536 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ina2xx.ko 2024-01-16T14:48:50.536 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/it87.ko 2024-01-16T14:48:50.536 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/k10temp.ko 2024-01-16T14:48:50.536 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/k8temp.ko 2024-01-16T14:48:50.537 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lineage-pem.ko 2024-01-16T14:48:50.537 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm63.ko 2024-01-16T14:48:50.537 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm73.ko 2024-01-16T14:48:50.537 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm75.ko 2024-01-16T14:48:50.537 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm77.ko 2024-01-16T14:48:50.537 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm78.ko 2024-01-16T14:48:50.537 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm80.ko 2024-01-16T14:48:50.537 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm83.ko 2024-01-16T14:48:50.537 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm85.ko 2024-01-16T14:48:50.537 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm87.ko 2024-01-16T14:48:50.537 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm90.ko 2024-01-16T14:48:50.537 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm92.ko 2024-01-16T14:48:50.537 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm93.ko 2024-01-16T14:48:50.537 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm95234.ko 2024-01-16T14:48:50.537 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm95241.ko 2024-01-16T14:48:50.537 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm95245.ko 2024-01-16T14:48:50.538 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc2945.ko 2024-01-16T14:48:50.538 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc4151.ko 2024-01-16T14:48:50.538 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc4215.ko 2024-01-16T14:48:50.538 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc4222.ko 2024-01-16T14:48:50.538 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc4245.ko 2024-01-16T14:48:50.538 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc4260.ko 2024-01-16T14:48:50.538 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc4261.ko 2024-01-16T14:48:50.538 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max16065.ko 2024-01-16T14:48:50.538 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max1619.ko 2024-01-16T14:48:50.538 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max1668.ko 2024-01-16T14:48:50.538 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max197.ko 2024-01-16T14:48:50.538 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max6639.ko 2024-01-16T14:48:50.538 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max6650.ko 2024-01-16T14:48:50.538 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max6697.ko 2024-01-16T14:48:50.538 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/mcp3021.ko 2024-01-16T14:48:50.538 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/nct6683.ko 2024-01-16T14:48:50.539 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/nct6775-core.ko 2024-01-16T14:48:50.539 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/nct6775.ko 2024-01-16T14:48:50.539 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/nct7802.ko 2024-01-16T14:48:50.539 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pc87360.ko 2024-01-16T14:48:50.539 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pc87427.ko 2024-01-16T14:48:50.539 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pcf8591.ko 2024-01-16T14:48:50.539 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus 2024-01-16T14:48:50.539 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-01-16T14:48:50.539 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-01-16T14:48:50.539 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-01-16T14:48:50.539 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/max16064.ko 2024-01-16T14:48:50.539 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/max34440.ko 2024-01-16T14:48:50.539 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/max8688.ko 2024-01-16T14:48:50.539 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-01-16T14:48:50.539 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-01-16T14:48:50.539 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-01-16T14:48:50.540 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-01-16T14:48:50.540 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-01-16T14:48:50.540 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-01-16T14:48:50.540 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/powr1220.ko 2024-01-16T14:48:50.540 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/sch5627.ko 2024-01-16T14:48:50.540 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/sch5636.ko 2024-01-16T14:48:50.540 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/sch56xx-common.ko 2024-01-16T14:48:50.540 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/sht15.ko 2024-01-16T14:48:50.540 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/sht21.ko 2024-01-16T14:48:50.540 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/shtc1.ko 2024-01-16T14:48:50.540 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/sis5595.ko 2024-01-16T14:48:50.540 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/smsc47b397.ko 2024-01-16T14:48:50.540 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/smsc47m1.ko 2024-01-16T14:48:50.540 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/smsc47m192.ko 2024-01-16T14:48:50.541 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/thmc50.ko 2024-01-16T14:48:50.541 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/tmp102.ko 2024-01-16T14:48:50.541 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/tmp103.ko 2024-01-16T14:48:50.541 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/tmp401.ko 2024-01-16T14:48:50.541 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/tmp421.ko 2024-01-16T14:48:50.541 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/via-cputemp.ko 2024-01-16T14:48:50.541 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/via686a.ko 2024-01-16T14:48:50.541 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/vt1211.ko 2024-01-16T14:48:50.541 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/vt8231.ko 2024-01-16T14:48:50.541 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83627ehf.ko 2024-01-16T14:48:50.541 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83627hf.ko 2024-01-16T14:48:50.541 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83781d.ko 2024-01-16T14:48:50.541 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83791d.ko 2024-01-16T14:48:50.541 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83792d.ko 2024-01-16T14:48:50.542 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83793.ko 2024-01-16T14:48:50.542 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83795.ko 2024-01-16T14:48:50.542 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83l785ts.ko 2024-01-16T14:48:50.542 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83l786ng.ko 2024-01-16T14:48:50.542 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c 2024-01-16T14:48:50.542 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/algos 2024-01-16T14:48:50.542 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-01-16T14:48:50.542 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-01-16T14:48:50.542 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses 2024-01-16T14:48:50.542 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-01-16T14:48:50.542 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-01-16T14:48:50.542 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-01-16T14:48:50.542 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-01-16T14:48:50.542 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-01-16T14:48:50.542 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-01-16T14:48:50.542 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-01-16T14:48:50.543 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-01-16T14:48:50.543 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-i801.ko 2024-01-16T14:48:50.543 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-isch.ko 2024-01-16T14:48:50.543 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-01-16T14:48:50.543 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-01-16T14:48:50.543 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-01-16T14:48:50.543 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-parport.ko 2024-01-16T14:48:50.543 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-01-16T14:48:50.543 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-01-16T14:48:50.543 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-01-16T14:48:50.543 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-01-16T14:48:50.543 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-01-16T14:48:50.543 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-01-16T14:48:50.543 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-via.ko 2024-01-16T14:48:50.543 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-01-16T14:48:50.543 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-01-16T14:48:50.544 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/i2c-dev.ko 2024-01-16T14:48:50.544 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/i2c-mux.ko 2024-01-16T14:48:50.544 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-01-16T14:48:50.544 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/i2c-smbus.ko 2024-01-16T14:48:50.544 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/i2c-stub.ko 2024-01-16T14:48:50.544 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband 2024-01-16T14:48:50.544 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core 2024-01-16T14:48:50.544 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/ib_cm.ko 2024-01-16T14:48:50.544 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/ib_core.ko 2024-01-16T14:48:50.544 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/ib_umad.ko 2024-01-16T14:48:50.544 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-01-16T14:48:50.544 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/iw_cm.ko 2024-01-16T14:48:50.544 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/rdma_cm.ko 2024-01-16T14:48:50.544 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-01-16T14:48:50.544 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw 2024-01-16T14:48:50.544 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/cxgb4 2024-01-16T14:48:50.545 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-01-16T14:48:50.545 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/mlx4 2024-01-16T14:48:50.545 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-01-16T14:48:50.545 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/mlx5 2024-01-16T14:48:50.545 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-01-16T14:48:50.545 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/mthca 2024-01-16T14:48:50.545 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-01-16T14:48:50.545 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp 2024-01-16T14:48:50.545 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/ipoib 2024-01-16T14:48:50.545 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-01-16T14:48:50.545 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/iser 2024-01-16T14:48:50.545 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-01-16T14:48:50.545 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/isert 2024-01-16T14:48:50.545 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-01-16T14:48:50.545 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/srp 2024-01-16T14:48:50.545 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-01-16T14:48:50.546 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/srpt 2024-01-16T14:48:50.546 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-01-16T14:48:50.546 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input 2024-01-16T14:48:50.546 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/gameport 2024-01-16T14:48:50.546 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-01-16T14:48:50.546 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/gameport/fm801-gp.ko 2024-01-16T14:48:50.546 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/gameport/gameport.ko 2024-01-16T14:48:50.546 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/joydev.ko 2024-01-16T14:48:50.546 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard 2024-01-16T14:48:50.546 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/gpio_keys.ko 2024-01-16T14:48:50.546 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse 2024-01-16T14:48:50.546 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/appletouch.ko 2024-01-16T14:48:50.546 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/bcm5974.ko 2024-01-16T14:48:50.546 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/cyapatp.ko 2024-01-16T14:48:50.546 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/elan_i2c.ko 2024-01-16T14:48:50.546 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/sermouse.ko 2024-01-16T14:48:50.547 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-01-16T14:48:50.547 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/synaptics_usb.ko 2024-01-16T14:48:50.547 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/vsxxxaa.ko 2024-01-16T14:48:50.547 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/rmi4 2024-01-16T14:48:50.547 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/rmi4/rmi_core.ko 2024-01-16T14:48:50.547 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio 2024-01-16T14:48:50.547 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio/altera_ps2.ko 2024-01-16T14:48:50.547 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio/arc_ps2.ko 2024-01-16T14:48:50.547 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-01-16T14:48:50.547 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio/serio_raw.ko 2024-01-16T14:48:50.547 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/sparse-keymap.ko 2024-01-16T14:48:50.547 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds 2024-01-16T14:48:50.547 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-blinkm.ko 2024-01-16T14:48:50.547 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-lm3530.ko 2024-01-16T14:48:50.547 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-lp3944.ko 2024-01-16T14:48:50.548 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-ss4200.ko 2024-01-16T14:48:50.548 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger 2024-01-16T14:48:50.548 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-01-16T14:48:50.548 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-01-16T14:48:50.548 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-01-16T14:48:50.548 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-01-16T14:48:50.548 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-01-16T14:48:50.548 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-01-16T14:48:50.548 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-01-16T14:48:50.548 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-01-16T14:48:50.548 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-01-16T14:48:50.548 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md 2024-01-16T14:48:50.548 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/bcache 2024-01-16T14:48:50.548 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/bcache/bcache.ko 2024-01-16T14:48:50.548 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-bio-prison.ko 2024-01-16T14:48:50.548 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-cache-smq.ko 2024-01-16T14:48:50.549 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-cache.ko 2024-01-16T14:48:50.549 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-crypt.ko 2024-01-16T14:48:50.549 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-delay.ko 2024-01-16T14:48:50.549 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-flakey.ko 2024-01-16T14:48:50.549 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-log-userspace.ko 2024-01-16T14:48:50.549 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-multipath.ko 2024-01-16T14:48:50.549 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-queue-length.ko 2024-01-16T14:48:50.549 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-raid.ko 2024-01-16T14:48:50.549 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-round-robin.ko 2024-01-16T14:48:50.549 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-service-time.ko 2024-01-16T14:48:50.549 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-switch.ko 2024-01-16T14:48:50.549 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-thin-pool.ko 2024-01-16T14:48:50.549 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-verity.ko 2024-01-16T14:48:50.549 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/faulty.ko 2024-01-16T14:48:50.549 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/linear.ko 2024-01-16T14:48:50.549 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/multipath.ko 2024-01-16T14:48:50.550 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/persistent-data 2024-01-16T14:48:50.550 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-01-16T14:48:50.550 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/raid0.ko 2024-01-16T14:48:50.550 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/raid1.ko 2024-01-16T14:48:50.550 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/raid10.ko 2024-01-16T14:48:50.550 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/raid456.ko 2024-01-16T14:48:50.550 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/message 2024-01-16T14:48:50.550 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion 2024-01-16T14:48:50.550 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptbase.ko 2024-01-16T14:48:50.550 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptctl.ko 2024-01-16T14:48:50.550 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptfc.ko 2024-01-16T14:48:50.550 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptlan.ko 2024-01-16T14:48:50.550 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptsas.ko 2024-01-16T14:48:50.550 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptscsih.ko 2024-01-16T14:48:50.550 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptspi.ko 2024-01-16T14:48:50.551 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd 2024-01-16T14:48:50.551 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/lpc_ich.ko 2024-01-16T14:48:50.551 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/lpc_sch.ko 2024-01-16T14:48:50.551 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/mfd-core.ko 2024-01-16T14:48:50.551 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/sm501.ko 2024-01-16T14:48:50.551 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/viperboard.ko 2024-01-16T14:48:50.551 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/vx855.ko 2024-01-16T14:48:50.551 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/wl1273-core.ko 2024-01-16T14:48:50.551 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc 2024-01-16T14:48:50.551 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/altera-stapl 2024-01-16T14:48:50.551 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-01-16T14:48:50.551 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/apds9802als.ko 2024-01-16T14:48:50.551 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/apds990x.ko 2024-01-16T14:48:50.551 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/bh1770glc.ko 2024-01-16T14:48:50.551 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/cb710 2024-01-16T14:48:50.551 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/cb710/cb710.ko 2024-01-16T14:48:50.552 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/eeprom 2024-01-16T14:48:50.552 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/eeprom/at24.ko 2024-01-16T14:48:50.552 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-01-16T14:48:50.552 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/eeprom/max6875.ko 2024-01-16T14:48:50.552 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/enclosure.ko 2024-01-16T14:48:50.552 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/hpilo.ko 2024-01-16T14:48:50.552 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/isl29003.ko 2024-01-16T14:48:50.552 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/isl29020.ko 2024-01-16T14:48:50.552 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/lis3lv02d 2024-01-16T14:48:50.552 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-01-16T14:48:50.552 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-01-16T14:48:50.552 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/mei 2024-01-16T14:48:50.552 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/mei/mei-me.ko 2024-01-16T14:48:50.552 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/mei/mei-txe.ko 2024-01-16T14:48:50.553 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/mei/mei.ko 2024-01-16T14:48:50.553 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/sgi-gru 2024-01-16T14:48:50.553 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/sgi-gru/gru.ko 2024-01-16T14:48:50.553 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/sgi-xp 2024-01-16T14:48:50.553 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/sgi-xp/xp.ko 2024-01-16T14:48:50.553 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/sgi-xp/xpc.ko 2024-01-16T14:48:50.553 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-01-16T14:48:50.553 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/tifm_7xx1.ko 2024-01-16T14:48:50.553 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/tifm_core.ko 2024-01-16T14:48:50.553 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/tsl2550.ko 2024-01-16T14:48:50.553 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/vmw_balloon.ko 2024-01-16T14:48:50.553 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/vmw_vmci 2024-01-16T14:48:50.553 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-01-16T14:48:50.553 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net 2024-01-16T14:48:50.553 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/bonding 2024-01-16T14:48:50.553 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/bonding/bonding.ko 2024-01-16T14:48:50.554 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa 2024-01-16T14:48:50.554 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa/b53 2024-01-16T14:48:50.554 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa/b53/b53_common.ko 2024-01-16T14:48:50.554 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa/bcm-sf2.ko 2024-01-16T14:48:50.554 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa/mv88e6060.ko 2024-01-16T14:48:50.554 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa/mv88e6xxx 2024-01-16T14:48:50.554 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-01-16T14:48:50.554 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dummy.ko 2024-01-16T14:48:50.554 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/eql.ko 2024-01-16T14:48:50.554 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet 2024-01-16T14:48:50.554 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/3com 2024-01-16T14:48:50.554 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-01-16T14:48:50.554 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-01-16T14:48:50.554 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-01-16T14:48:50.554 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-01-16T14:48:50.554 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/8390 2024-01-16T14:48:50.555 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/8390/8390.ko 2024-01-16T14:48:50.555 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-01-16T14:48:50.555 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-01-16T14:48:50.555 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-01-16T14:48:50.555 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/adaptec 2024-01-16T14:48:50.555 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-01-16T14:48:50.555 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/agere 2024-01-16T14:48:50.555 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/agere/et131x.ko 2024-01-16T14:48:50.555 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/alteon 2024-01-16T14:48:50.555 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-01-16T14:48:50.555 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/altera 2024-01-16T14:48:50.555 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-01-16T14:48:50.555 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/amd 2024-01-16T14:48:50.555 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-01-16T14:48:50.555 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-01-16T14:48:50.556 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-01-16T14:48:50.556 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros 2024-01-16T14:48:50.556 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/alx 2024-01-16T14:48:50.556 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-01-16T14:48:50.556 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atl1c 2024-01-16T14:48:50.556 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-01-16T14:48:50.556 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atl1e 2024-01-16T14:48:50.556 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-01-16T14:48:50.556 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atlx 2024-01-16T14:48:50.556 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-01-16T14:48:50.556 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-01-16T14:48:50.556 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom 2024-01-16T14:48:50.556 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-01-16T14:48:50.557 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-01-16T14:48:50.557 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-01-16T14:48:50.557 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-01-16T14:48:50.557 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-01-16T14:48:50.557 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/genet 2024-01-16T14:48:50.557 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-01-16T14:48:50.557 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-01-16T14:48:50.557 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/brocade 2024-01-16T14:48:50.557 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/brocade/bna 2024-01-16T14:48:50.557 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-01-16T14:48:50.557 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio 2024-01-16T14:48:50.557 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb 2024-01-16T14:48:50.557 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-01-16T14:48:50.557 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-01-16T14:48:50.558 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-01-16T14:48:50.558 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-01-16T14:48:50.558 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-01-16T14:48:50.558 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-01-16T14:48:50.558 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-01-16T14:48:50.558 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-01-16T14:48:50.558 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-01-16T14:48:50.558 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/cisco 2024-01-16T14:48:50.558 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/cisco/enic 2024-01-16T14:48:50.558 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-01-16T14:48:50.558 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec 2024-01-16T14:48:50.558 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip 2024-01-16T14:48:50.558 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-01-16T14:48:50.558 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-01-16T14:48:50.558 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-01-16T14:48:50.559 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-01-16T14:48:50.559 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-01-16T14:48:50.559 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-01-16T14:48:50.559 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dlink 2024-01-16T14:48:50.559 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-01-16T14:48:50.559 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-01-16T14:48:50.559 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dnet.ko 2024-01-16T14:48:50.559 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/emulex 2024-01-16T14:48:50.559 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/emulex/benet 2024-01-16T14:48:50.559 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-01-16T14:48:50.559 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/ethoc.ko 2024-01-16T14:48:50.559 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/fealnx.ko 2024-01-16T14:48:50.559 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel 2024-01-16T14:48:50.559 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/e100.ko 2024-01-16T14:48:50.559 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/e1000 2024-01-16T14:48:50.559 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-01-16T14:48:50.560 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/e1000e 2024-01-16T14:48:50.560 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-01-16T14:48:50.560 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/fm10k 2024-01-16T14:48:50.560 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-01-16T14:48:50.560 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/i40e 2024-01-16T14:48:50.560 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-01-16T14:48:50.560 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/iavf 2024-01-16T14:48:50.560 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-01-16T14:48:50.560 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/igb 2024-01-16T14:48:50.560 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-01-16T14:48:50.560 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/igbvf 2024-01-16T14:48:50.560 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-01-16T14:48:50.560 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/ixgbe 2024-01-16T14:48:50.560 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-01-16T14:48:50.560 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/ixgbevf 2024-01-16T14:48:50.561 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-01-16T14:48:50.561 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/jme.ko 2024-01-16T14:48:50.561 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/marvell 2024-01-16T14:48:50.561 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-01-16T14:48:50.561 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/marvell/skge.ko 2024-01-16T14:48:50.561 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-01-16T14:48:50.561 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox 2024-01-16T14:48:50.561 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox/mlx4 2024-01-16T14:48:50.561 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-01-16T14:48:50.561 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-01-16T14:48:50.561 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox/mlx5 2024-01-16T14:48:50.561 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-01-16T14:48:50.561 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-01-16T14:48:50.561 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/micrel 2024-01-16T14:48:50.561 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-01-16T14:48:50.561 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/myricom 2024-01-16T14:48:50.562 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/myricom/myri10ge 2024-01-16T14:48:50.562 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-01-16T14:48:50.562 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/natsemi 2024-01-16T14:48:50.562 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-01-16T14:48:50.562 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-01-16T14:48:50.562 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/neterion 2024-01-16T14:48:50.562 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-01-16T14:48:50.562 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/nvidia 2024-01-16T14:48:50.562 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-01-16T14:48:50.562 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/packetengines 2024-01-16T14:48:50.562 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-01-16T14:48:50.562 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-01-16T14:48:50.562 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/qlogic 2024-01-16T14:48:50.562 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/qlogic/netxen 2024-01-16T14:48:50.562 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-01-16T14:48:50.562 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-01-16T14:48:50.563 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-01-16T14:48:50.563 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-01-16T14:48:50.563 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/rdc 2024-01-16T14:48:50.563 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-01-16T14:48:50.563 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/realtek 2024-01-16T14:48:50.563 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-01-16T14:48:50.563 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-01-16T14:48:50.563 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/realtek/atp.ko 2024-01-16T14:48:50.563 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-01-16T14:48:50.563 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/rocker 2024-01-16T14:48:50.563 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-01-16T14:48:50.563 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sfc 2024-01-16T14:48:50.563 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-01-16T14:48:50.563 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/silan 2024-01-16T14:48:50.563 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-01-16T14:48:50.563 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sis 2024-01-16T14:48:50.564 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sis/sis190.ko 2024-01-16T14:48:50.564 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sis/sis900.ko 2024-01-16T14:48:50.564 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/smsc 2024-01-16T14:48:50.564 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-01-16T14:48:50.564 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-01-16T14:48:50.564 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-01-16T14:48:50.564 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-01-16T14:48:50.564 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/stmicro 2024-01-16T14:48:50.564 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/stmicro/stmmac 2024-01-16T14:48:50.564 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-01-16T14:48:50.564 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-01-16T14:48:50.564 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sun 2024-01-16T14:48:50.564 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sun/cassini.ko 2024-01-16T14:48:50.564 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sun/niu.ko 2024-01-16T14:48:50.564 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sun/sungem.ko 2024-01-16T14:48:50.564 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-01-16T14:48:50.565 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/tehuti 2024-01-16T14:48:50.565 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-01-16T14:48:50.565 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/ti 2024-01-16T14:48:50.565 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/ti/tlan.ko 2024-01-16T14:48:50.565 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/via 2024-01-16T14:48:50.565 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-01-16T14:48:50.565 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-01-16T14:48:50.565 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/wiznet 2024-01-16T14:48:50.565 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-01-16T14:48:50.565 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-01-16T14:48:50.565 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/xircom 2024-01-16T14:48:50.565 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-01-16T14:48:50.565 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/geneve.ko 2024-01-16T14:48:50.565 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/hyperv 2024-01-16T14:48:50.565 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-01-16T14:48:50.565 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ieee802154 2024-01-16T14:48:50.566 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ieee802154/fakelb.ko 2024-01-16T14:48:50.566 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ifb.ko 2024-01-16T14:48:50.566 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ipvlan 2024-01-16T14:48:50.566 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ipvlan/ipvlan.ko 2024-01-16T14:48:50.566 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/macvlan.ko 2024-01-16T14:48:50.566 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/macvtap.ko 2024-01-16T14:48:50.566 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/mdio 2024-01-16T14:48:50.566 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/mdio.ko 2024-01-16T14:48:50.566 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-01-16T14:48:50.566 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-01-16T14:48:50.566 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/mdio/mdio-regmap.ko 2024-01-16T14:48:50.566 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/mii.ko 2024-01-16T14:48:50.566 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/net_failover.ko 2024-01-16T14:48:50.566 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/netconsole.ko 2024-01-16T14:48:50.566 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/nlmon.ko 2024-01-16T14:48:50.566 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/pcs 2024-01-16T14:48:50.567 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/pcs/pcs-lynx.ko 2024-01-16T14:48:50.567 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-01-16T14:48:50.567 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy 2024-01-16T14:48:50.567 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/amd.ko 2024-01-16T14:48:50.567 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-01-16T14:48:50.567 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-01-16T14:48:50.567 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/bcm7xxx.ko 2024-01-16T14:48:50.567 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/bcm87xx.ko 2024-01-16T14:48:50.567 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/broadcom.ko 2024-01-16T14:48:50.567 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/cicada.ko 2024-01-16T14:48:50.567 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/davicom.ko 2024-01-16T14:48:50.567 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/dp83640.ko 2024-01-16T14:48:50.567 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/et1011c.ko 2024-01-16T14:48:50.567 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/icplus.ko 2024-01-16T14:48:50.567 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/lxt.ko 2024-01-16T14:48:50.567 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/marvell.ko 2024-01-16T14:48:50.568 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/micrel.ko 2024-01-16T14:48:50.568 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/national.ko 2024-01-16T14:48:50.568 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/phylink.ko 2024-01-16T14:48:50.568 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/qsemi.ko 2024-01-16T14:48:50.568 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/realtek.ko 2024-01-16T14:48:50.568 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/smsc.ko 2024-01-16T14:48:50.568 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/ste10Xp.ko 2024-01-16T14:48:50.568 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/vitesse.ko 2024-01-16T14:48:50.568 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp 2024-01-16T14:48:50.568 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/bsd_comp.ko 2024-01-16T14:48:50.568 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/ppp_async.ko 2024-01-16T14:48:50.568 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/ppp_deflate.ko 2024-01-16T14:48:50.568 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/ppp_generic.ko 2024-01-16T14:48:50.568 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/ppp_mppe.ko 2024-01-16T14:48:50.568 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/ppp_synctty.ko 2024-01-16T14:48:50.569 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/pppoe.ko 2024-01-16T14:48:50.569 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/pppox.ko 2024-01-16T14:48:50.569 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/pptp.ko 2024-01-16T14:48:50.569 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/slip 2024-01-16T14:48:50.569 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/slip/slhc.ko 2024-01-16T14:48:50.569 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/slip/slip.ko 2024-01-16T14:48:50.569 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/sungem_phy.ko 2024-01-16T14:48:50.569 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/tap.ko 2024-01-16T14:48:50.569 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team 2024-01-16T14:48:50.569 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team/team.ko 2024-01-16T14:48:50.569 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team/team_mode_activebackup.ko 2024-01-16T14:48:50.569 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team/team_mode_broadcast.ko 2024-01-16T14:48:50.569 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-01-16T14:48:50.569 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team/team_mode_random.ko 2024-01-16T14:48:50.569 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-01-16T14:48:50.569 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/tun.ko 2024-01-16T14:48:50.569 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/veth.ko 2024-01-16T14:48:50.570 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/virtio_net.ko 2024-01-16T14:48:50.570 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/vmxnet3 2024-01-16T14:48:50.570 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-01-16T14:48:50.570 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/vxlan 2024-01-16T14:48:50.570 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/vxlan/vxlan.ko 2024-01-16T14:48:50.570 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/xen-netback 2024-01-16T14:48:50.570 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/xen-netback/xen-netback.ko 2024-01-16T14:48:50.570 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/xen-netfront.ko 2024-01-16T14:48:50.570 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ntb 2024-01-16T14:48:50.570 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ntb/ntb.ko 2024-01-16T14:48:50.570 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/nvme 2024-01-16T14:48:50.570 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/nvme/host 2024-01-16T14:48:50.570 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/nvme/host/nvme-core.ko 2024-01-16T14:48:50.570 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/nvme/host/nvme.ko 2024-01-16T14:48:50.570 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/parport 2024-01-16T14:48:50.571 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/parport/parport.ko 2024-01-16T14:48:50.571 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/parport/parport_cs.ko 2024-01-16T14:48:50.571 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/parport/parport_pc.ko 2024-01-16T14:48:50.571 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/parport/parport_serial.ko 2024-01-16T14:48:50.571 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pci 2024-01-16T14:48:50.571 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pci/hotplug 2024-01-16T14:48:50.571 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-01-16T14:48:50.571 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pci/pcie 2024-01-16T14:48:50.571 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pci/pcie/aer_inject.ko 2024-01-16T14:48:50.571 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pci/xen-pcifront.ko 2024-01-16T14:48:50.571 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pcmcia 2024-01-16T14:48:50.571 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pcmcia/i82092.ko 2024-01-16T14:48:50.571 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pcmcia/pd6729.ko 2024-01-16T14:48:50.571 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pcmcia/yenta_socket.ko 2024-01-16T14:48:50.571 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pinctrl 2024-01-16T14:48:50.571 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pinctrl/intel 2024-01-16T14:48:50.572 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-01-16T14:48:50.572 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform 2024-01-16T14:48:50.572 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86 2024-01-16T14:48:50.572 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/acer-wmi.ko 2024-01-16T14:48:50.572 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/acerhdf.ko 2024-01-16T14:48:50.572 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/apple-gmux.ko 2024-01-16T14:48:50.572 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/asus-laptop.ko 2024-01-16T14:48:50.572 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-01-16T14:48:50.572 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/asus-wmi.ko 2024-01-16T14:48:50.572 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/classmate-laptop.ko 2024-01-16T14:48:50.572 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-01-16T14:48:50.572 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-01-16T14:48:50.572 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-01-16T14:48:50.572 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-01-16T14:48:50.572 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/hdaps.ko 2024-01-16T14:48:50.572 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/intel 2024-01-16T14:48:50.573 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-01-16T14:48:50.573 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/intel_ips.ko 2024-01-16T14:48:50.573 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/msi-wmi.ko 2024-01-16T14:48:50.573 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/mxm-wmi.ko 2024-01-16T14:48:50.573 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-01-16T14:48:50.573 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/samsung-laptop.ko 2024-01-16T14:48:50.573 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/samsung-q10.ko 2024-01-16T14:48:50.573 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/topstar-laptop.ko 2024-01-16T14:48:50.573 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-01-16T14:48:50.573 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/toshiba_haps.ko 2024-01-16T14:48:50.573 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/wmi-bmof.ko 2024-01-16T14:48:50.573 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/wmi.ko 2024-01-16T14:48:50.573 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/powercap 2024-01-16T14:48:50.573 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/powercap/intel_rapl_common.ko 2024-01-16T14:48:50.573 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/powercap/intel_rapl_msr.ko 2024-01-16T14:48:50.573 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pps 2024-01-16T14:48:50.574 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pps/clients 2024-01-16T14:48:50.574 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pps/clients/pps-gpio.ko 2024-01-16T14:48:50.574 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pps/clients/pps-ldisc.ko 2024-01-16T14:48:50.574 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pps/clients/pps_parport.ko 2024-01-16T14:48:50.574 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pps/pps_core.ko 2024-01-16T14:48:50.574 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ptp 2024-01-16T14:48:50.574 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ptp/ptp.ko 2024-01-16T14:48:50.574 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ptp/ptp_kvm.ko 2024-01-16T14:48:50.574 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc 2024-01-16T14:48:50.574 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-bq32k.ko 2024-01-16T14:48:50.574 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1286.ko 2024-01-16T14:48:50.574 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1307.ko 2024-01-16T14:48:50.574 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1374.ko 2024-01-16T14:48:50.574 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1511.ko 2024-01-16T14:48:50.574 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1553.ko 2024-01-16T14:48:50.575 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1672.ko 2024-01-16T14:48:50.575 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1742.ko 2024-01-16T14:48:50.575 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds2404.ko 2024-01-16T14:48:50.575 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds3232.ko 2024-01-16T14:48:50.575 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-em3027.ko 2024-01-16T14:48:50.575 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-fm3130.ko 2024-01-16T14:48:50.575 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-isl12022.ko 2024-01-16T14:48:50.575 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-isl1208.ko 2024-01-16T14:48:50.575 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-m41t80.ko 2024-01-16T14:48:50.575 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-m48t35.ko 2024-01-16T14:48:50.575 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-m48t59.ko 2024-01-16T14:48:50.575 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-max6900.ko 2024-01-16T14:48:50.575 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-msm6242.ko 2024-01-16T14:48:50.575 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-pcf2127.ko 2024-01-16T14:48:50.575 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-pcf85063.ko 2024-01-16T14:48:50.576 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-pcf8523.ko 2024-01-16T14:48:50.576 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-pcf8563.ko 2024-01-16T14:48:50.576 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-pcf8583.ko 2024-01-16T14:48:50.576 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-rp5c01.ko 2024-01-16T14:48:50.576 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-rs5c372.ko 2024-01-16T14:48:50.576 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-01-16T14:48:50.576 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-rx8025.ko 2024-01-16T14:48:50.576 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-rx8581.ko 2024-01-16T14:48:50.576 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-01-16T14:48:50.576 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-x1205.ko 2024-01-16T14:48:50.576 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi 2024-01-16T14:48:50.576 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/3w-9xxx.ko 2024-01-16T14:48:50.576 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/3w-sas.ko 2024-01-16T14:48:50.576 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/3w-xxxx.ko 2024-01-16T14:48:50.576 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/BusLogic.ko 2024-01-16T14:48:50.577 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/a100u2w.ko 2024-01-16T14:48:50.577 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/aacraid 2024-01-16T14:48:50.577 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/aacraid/aacraid.ko 2024-01-16T14:48:50.577 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/advansys.ko 2024-01-16T14:48:50.577 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/aic7xxx 2024-01-16T14:48:50.577 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-01-16T14:48:50.577 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-01-16T14:48:50.577 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/am53c974.ko 2024-01-16T14:48:50.577 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/arcmsr 2024-01-16T14:48:50.577 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-01-16T14:48:50.577 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/atp870u.ko 2024-01-16T14:48:50.577 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/be2iscsi 2024-01-16T14:48:50.577 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-01-16T14:48:50.577 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/bfa 2024-01-16T14:48:50.577 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/bfa/bfa.ko 2024-01-16T14:48:50.577 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/bnx2fc 2024-01-16T14:48:50.578 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-01-16T14:48:50.578 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/bnx2i 2024-01-16T14:48:50.578 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-01-16T14:48:50.578 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/ch.ko 2024-01-16T14:48:50.578 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/csiostor 2024-01-16T14:48:50.578 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/csiostor/csiostor.ko 2024-01-16T14:48:50.578 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/cxgbi 2024-01-16T14:48:50.578 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/cxgbi/cxgb3i 2024-01-16T14:48:50.578 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-01-16T14:48:50.578 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/cxgbi/cxgb4i 2024-01-16T14:48:50.578 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-01-16T14:48:50.578 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-01-16T14:48:50.578 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/dc395x.ko 2024-01-16T14:48:50.578 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/device_handler 2024-01-16T14:48:50.578 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-01-16T14:48:50.579 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-01-16T14:48:50.579 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-01-16T14:48:50.579 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-01-16T14:48:50.579 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/dmx3191d.ko 2024-01-16T14:48:50.579 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/esas2r 2024-01-16T14:48:50.579 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/esas2r/esas2r.ko 2024-01-16T14:48:50.579 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/esp_scsi.ko 2024-01-16T14:48:50.579 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/fcoe 2024-01-16T14:48:50.579 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/fcoe/fcoe.ko 2024-01-16T14:48:50.579 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-01-16T14:48:50.579 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/fnic 2024-01-16T14:48:50.579 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/fnic/fnic.ko 2024-01-16T14:48:50.579 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/hpsa.ko 2024-01-16T14:48:50.579 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/hptiop.ko 2024-01-16T14:48:50.579 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/hv_storvsc.ko 2024-01-16T14:48:50.580 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/initio.ko 2024-01-16T14:48:50.580 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/ipr.ko 2024-01-16T14:48:50.580 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/ips.ko 2024-01-16T14:48:50.580 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/isci 2024-01-16T14:48:50.580 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/isci/isci.ko 2024-01-16T14:48:50.580 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-01-16T14:48:50.580 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/iscsi_tcp.ko 2024-01-16T14:48:50.580 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/libfc 2024-01-16T14:48:50.580 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/libfc/libfc.ko 2024-01-16T14:48:50.580 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/libiscsi.ko 2024-01-16T14:48:50.580 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/libiscsi_tcp.ko 2024-01-16T14:48:50.580 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/libsas 2024-01-16T14:48:50.580 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/libsas/libsas.ko 2024-01-16T14:48:50.580 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/lpfc 2024-01-16T14:48:50.581 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/lpfc/lpfc.ko 2024-01-16T14:48:50.581 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/megaraid 2024-01-16T14:48:50.581 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/megaraid.ko 2024-01-16T14:48:50.581 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-01-16T14:48:50.581 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-01-16T14:48:50.581 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-01-16T14:48:50.581 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/mpt3sas 2024-01-16T14:48:50.581 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-01-16T14:48:50.581 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/mvsas 2024-01-16T14:48:50.581 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/mvsas/mvsas.ko 2024-01-16T14:48:50.581 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/mvumi.ko 2024-01-16T14:48:50.581 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/pm8001 2024-01-16T14:48:50.581 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-01-16T14:48:50.581 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/pmcraid.ko 2024-01-16T14:48:50.581 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/qla1280.ko 2024-01-16T14:48:50.581 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/qla2xxx 2024-01-16T14:48:50.581 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-01-16T14:48:50.582 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-01-16T14:48:50.582 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/qla4xxx 2024-01-16T14:48:50.582 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-01-16T14:48:50.582 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/raid_class.ko 2024-01-16T14:48:50.582 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/scsi_debug.ko 2024-01-16T14:48:50.582 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/scsi_transport_fc.ko 2024-01-16T14:48:50.582 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-01-16T14:48:50.582 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/scsi_transport_sas.ko 2024-01-16T14:48:50.582 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/scsi_transport_spi.ko 2024-01-16T14:48:50.582 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/scsi_transport_srp.ko 2024-01-16T14:48:50.582 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/ses.ko 2024-01-16T14:48:50.582 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/st.ko 2024-01-16T14:48:50.582 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/stex.ko 2024-01-16T14:48:50.582 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/sym53c8xx_2 2024-01-16T14:48:50.583 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-01-16T14:48:50.583 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/virtio_scsi.ko 2024-01-16T14:48:50.583 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/vmw_pvscsi.ko 2024-01-16T14:48:50.583 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/wd719x.ko 2024-01-16T14:48:50.583 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ssb 2024-01-16T14:48:50.583 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ssb/ssb.ko 2024-01-16T14:48:50.583 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target 2024-01-16T14:48:50.583 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/iscsi 2024-01-16T14:48:50.583 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-01-16T14:48:50.583 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/loopback 2024-01-16T14:48:50.583 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/loopback/tcm_loop.ko 2024-01-16T14:48:50.583 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/target_core_file.ko 2024-01-16T14:48:50.583 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/target_core_iblock.ko 2024-01-16T14:48:50.583 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/target_core_mod.ko 2024-01-16T14:48:50.583 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/target_core_pscsi.ko 2024-01-16T14:48:50.583 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/target_core_user.ko 2024-01-16T14:48:50.584 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/tcm_fc 2024-01-16T14:48:50.584 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-01-16T14:48:50.584 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal 2024-01-16T14:48:50.584 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel 2024-01-16T14:48:50.584 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal 2024-01-16T14:48:50.584 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-01-16T14:48:50.584 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-01-16T14:48:50.584 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-01-16T14:48:50.584 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-01-16T14:48:50.584 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-01-16T14:48:50.584 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-01-16T14:48:50.584 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-01-16T14:48:50.584 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-01-16T14:48:50.584 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-01-16T14:48:50.584 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-01-16T14:48:50.585 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-01-16T14:48:50.585 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-01-16T14:48:50.585 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-01-16T14:48:50.585 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-01-16T14:48:50.585 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-01-16T14:48:50.585 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-01-16T14:48:50.585 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-01-16T14:48:50.585 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-01-16T14:48:50.585 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty 2024-01-16T14:48:50.585 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/ipwireless 2024-01-16T14:48:50.585 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-01-16T14:48:50.585 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/n_gsm.ko 2024-01-16T14:48:50.585 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/n_hdlc.ko 2024-01-16T14:48:50.585 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/nozomi.ko 2024-01-16T14:48:50.585 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial 2024-01-16T14:48:50.585 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/8250 2024-01-16T14:48:50.586 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-01-16T14:48:50.586 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/arc_uart.ko 2024-01-16T14:48:50.586 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/jsm 2024-01-16T14:48:50.586 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/jsm/jsm.ko 2024-01-16T14:48:50.586 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ufs 2024-01-16T14:48:50.586 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ufs/core 2024-01-16T14:48:50.586 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ufs/core/ufshcd-core.ko 2024-01-16T14:48:50.586 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ufs/host 2024-01-16T14:48:50.586 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-01-16T14:48:50.586 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio 2024-01-16T14:48:50.586 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/uio.ko 2024-01-16T14:48:50.586 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/uio_aec.ko 2024-01-16T14:48:50.586 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/uio_cif.ko 2024-01-16T14:48:50.586 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/uio_pci_generic.ko 2024-01-16T14:48:50.586 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/uio_sercos3.ko 2024-01-16T14:48:50.586 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb 2024-01-16T14:48:50.587 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/class 2024-01-16T14:48:50.587 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/class/cdc-acm.ko 2024-01-16T14:48:50.587 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/class/cdc-wdm.ko 2024-01-16T14:48:50.587 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/class/usblp.ko 2024-01-16T14:48:50.587 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/class/usbtmc.ko 2024-01-16T14:48:50.587 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/host 2024-01-16T14:48:50.587 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/host/sl811-hcd.ko 2024-01-16T14:48:50.587 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/image 2024-01-16T14:48:50.587 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/image/mdc800.ko 2024-01-16T14:48:50.587 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/image/microtek.ko 2024-01-16T14:48:50.587 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc 2024-01-16T14:48:50.587 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/adutux.ko 2024-01-16T14:48:50.587 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/appledisplay.ko 2024-01-16T14:48:50.587 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/emi26.ko 2024-01-16T14:48:50.587 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/emi62.ko 2024-01-16T14:48:50.588 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/ezusb.ko 2024-01-16T14:48:50.588 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/idmouse.ko 2024-01-16T14:48:50.588 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/iowarrior.ko 2024-01-16T14:48:50.588 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/isight_firmware.ko 2024-01-16T14:48:50.588 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/ldusb.ko 2024-01-16T14:48:50.588 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/legousbtower.ko 2024-01-16T14:48:50.588 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/sisusbvga 2024-01-16T14:48:50.588 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-01-16T14:48:50.588 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/trancevibrator.ko 2024-01-16T14:48:50.588 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/usb3503.ko 2024-01-16T14:48:50.588 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/usblcd.ko 2024-01-16T14:48:50.588 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/usbsevseg.ko 2024-01-16T14:48:50.588 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/uss720.ko 2024-01-16T14:48:50.588 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/yurex.ko 2024-01-16T14:48:50.588 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/phy 2024-01-16T14:48:50.589 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/phy/phy-generic.ko 2024-01-16T14:48:50.589 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial 2024-01-16T14:48:50.589 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/aircable.ko 2024-01-16T14:48:50.589 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ark3116.ko 2024-01-16T14:48:50.589 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/belkin_sa.ko 2024-01-16T14:48:50.589 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ch341.ko 2024-01-16T14:48:50.589 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/cp210x.ko 2024-01-16T14:48:50.589 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/cyberjack.ko 2024-01-16T14:48:50.589 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/cypress_m8.ko 2024-01-16T14:48:50.589 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/digi_acceleport.ko 2024-01-16T14:48:50.589 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/empeg.ko 2024-01-16T14:48:50.589 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ftdi_sio.ko 2024-01-16T14:48:50.589 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/garmin_gps.ko 2024-01-16T14:48:50.589 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/io_edgeport.ko 2024-01-16T14:48:50.590 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/io_ti.ko 2024-01-16T14:48:50.590 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ipaq.ko 2024-01-16T14:48:50.590 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ipw.ko 2024-01-16T14:48:50.590 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ir-usb.ko 2024-01-16T14:48:50.590 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-01-16T14:48:50.590 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/keyspan.ko 2024-01-16T14:48:50.590 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/keyspan_pda.ko 2024-01-16T14:48:50.590 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/kl5kusb105.ko 2024-01-16T14:48:50.590 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/kobil_sct.ko 2024-01-16T14:48:50.590 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/mct_u232.ko 2024-01-16T14:48:50.590 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/mos7720.ko 2024-01-16T14:48:50.590 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/mos7840.ko 2024-01-16T14:48:50.590 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/navman.ko 2024-01-16T14:48:50.590 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/omninet.ko 2024-01-16T14:48:50.590 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/opticon.ko 2024-01-16T14:48:50.591 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/option.ko 2024-01-16T14:48:50.591 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/oti6858.ko 2024-01-16T14:48:50.591 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/pl2303.ko 2024-01-16T14:48:50.591 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/qcaux.ko 2024-01-16T14:48:50.591 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/qcserial.ko 2024-01-16T14:48:50.591 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/quatech2.ko 2024-01-16T14:48:50.591 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/safe_serial.ko 2024-01-16T14:48:50.591 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/sierra.ko 2024-01-16T14:48:50.591 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/spcp8x5.ko 2024-01-16T14:48:50.591 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ssu100.ko 2024-01-16T14:48:50.591 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/symbolserial.ko 2024-01-16T14:48:50.591 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-01-16T14:48:50.591 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-01-16T14:48:50.591 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/usb_debug.ko 2024-01-16T14:48:50.591 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/usb_wwan.ko 2024-01-16T14:48:50.591 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/visor.ko 2024-01-16T14:48:50.592 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/whiteheat.ko 2024-01-16T14:48:50.592 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/xsens_mt.ko 2024-01-16T14:48:50.592 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage 2024-01-16T14:48:50.592 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/uas.ko 2024-01-16T14:48:50.592 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-alauda.ko 2024-01-16T14:48:50.592 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-cypress.ko 2024-01-16T14:48:50.592 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-datafab.ko 2024-01-16T14:48:50.592 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-01-16T14:48:50.592 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-freecom.ko 2024-01-16T14:48:50.592 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-isd200.ko 2024-01-16T14:48:50.592 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-01-16T14:48:50.592 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-karma.ko 2024-01-16T14:48:50.592 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-onetouch.ko 2024-01-16T14:48:50.592 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-realtek.ko 2024-01-16T14:48:50.592 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-sddr09.ko 2024-01-16T14:48:50.593 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-sddr55.ko 2024-01-16T14:48:50.593 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-usbat.ko 2024-01-16T14:48:50.593 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/usb-storage.ko 2024-01-16T14:48:50.593 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/usbip 2024-01-16T14:48:50.593 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/usbip/usbip-core.ko 2024-01-16T14:48:50.593 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/usbip/usbip-host.ko 2024-01-16T14:48:50.593 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-01-16T14:48:50.593 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vfio 2024-01-16T14:48:50.593 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vfio/pci 2024-01-16T14:48:50.593 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-01-16T14:48:50.593 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vfio/pci/vfio-pci.ko 2024-01-16T14:48:50.593 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vfio/vfio.ko 2024-01-16T14:48:50.593 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-01-16T14:48:50.593 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vhost 2024-01-16T14:48:50.594 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vhost/vhost.ko 2024-01-16T14:48:50.594 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vhost/vhost_iotlb.ko 2024-01-16T14:48:50.594 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vhost/vhost_net.ko 2024-01-16T14:48:50.594 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vhost/vhost_scsi.ko 2024-01-16T14:48:50.594 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/video 2024-01-16T14:48:50.594 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight 2024-01-16T14:48:50.594 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/apple_bl.ko 2024-01-16T14:48:50.594 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/lcd.ko 2024-01-16T14:48:50.594 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/platform_lcd.ko 2024-01-16T14:48:50.594 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/virtio 2024-01-16T14:48:50.594 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/virtio/virtio.ko 2024-01-16T14:48:50.594 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/virtio/virtio_balloon.ko 2024-01-16T14:48:50.594 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/virtio/virtio_mmio.ko 2024-01-16T14:48:50.594 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/virtio/virtio_pci.ko 2024-01-16T14:48:50.595 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-01-16T14:48:50.595 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-01-16T14:48:50.595 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/virtio/virtio_ring.ko 2024-01-16T14:48:50.595 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1 2024-01-16T14:48:50.595 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/masters 2024-01-16T14:48:50.595 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/masters/ds2482.ko 2024-01-16T14:48:50.595 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/masters/ds2490.ko 2024-01-16T14:48:50.595 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves 2024-01-16T14:48:50.595 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-01-16T14:48:50.595 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-01-16T14:48:50.595 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-01-16T14:48:50.595 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-01-16T14:48:50.595 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-01-16T14:48:50.595 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-01-16T14:48:50.595 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-01-16T14:48:50.596 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-01-16T14:48:50.596 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-01-16T14:48:50.596 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_smem.ko 2024-01-16T14:48:50.596 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_therm.ko 2024-01-16T14:48:50.596 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/wire.ko 2024-01-16T14:48:50.596 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog 2024-01-16T14:48:50.596 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/alim1535_wdt.ko 2024-01-16T14:48:50.596 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/alim7101_wdt.ko 2024-01-16T14:48:50.596 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/f71808e_wdt.ko 2024-01-16T14:48:50.596 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/hpwdt.ko 2024-01-16T14:48:50.596 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/i6300esb.ko 2024-01-16T14:48:50.596 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-01-16T14:48:50.596 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/iTCO_wdt.ko 2024-01-16T14:48:50.597 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/ib700wdt.ko 2024-01-16T14:48:50.597 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/ibmasr.ko 2024-01-16T14:48:50.597 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-01-16T14:48:50.597 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/it8712f_wdt.ko 2024-01-16T14:48:50.597 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/it87_wdt.ko 2024-01-16T14:48:50.597 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/machzwd.ko 2024-01-16T14:48:50.597 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/nv_tco.ko 2024-01-16T14:48:50.597 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/pcwd_pci.ko 2024-01-16T14:48:50.597 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/pcwd_usb.ko 2024-01-16T14:48:50.597 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-01-16T14:48:50.597 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/sch311x_wdt.ko 2024-01-16T14:48:50.597 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/softdog.ko 2024-01-16T14:48:50.597 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/sp5100_tco.ko 2024-01-16T14:48:50.597 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/via_wdt.ko 2024-01-16T14:48:50.598 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-01-16T14:48:50.598 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/w83877f_wdt.ko 2024-01-16T14:48:50.598 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/w83977f_wdt.ko 2024-01-16T14:48:50.598 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/wdt_pci.ko 2024-01-16T14:48:50.598 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/xen_wdt.ko 2024-01-16T14:48:50.598 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen 2024-01-16T14:48:50.598 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-acpi-processor.ko 2024-01-16T14:48:50.598 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-evtchn.ko 2024-01-16T14:48:50.598 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-gntalloc.ko 2024-01-16T14:48:50.598 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-gntdev.ko 2024-01-16T14:48:50.598 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-pciback 2024-01-16T14:48:50.598 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-01-16T14:48:50.598 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-privcmd.ko 2024-01-16T14:48:50.598 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xenfs 2024-01-16T14:48:50.598 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xenfs/xenfs.ko 2024-01-16T14:48:50.599 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs 2024-01-16T14:48:50.599 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/9p 2024-01-16T14:48:50.599 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/9p/9p.ko 2024-01-16T14:48:50.599 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/affs 2024-01-16T14:48:50.599 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/affs/affs.ko 2024-01-16T14:48:50.599 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/befs 2024-01-16T14:48:50.599 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/befs/befs.ko 2024-01-16T14:48:50.599 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/binfmt_misc.ko 2024-01-16T14:48:50.599 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/btrfs 2024-01-16T14:48:50.599 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/btrfs/btrfs.ko 2024-01-16T14:48:50.599 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/cachefiles 2024-01-16T14:48:50.599 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/cachefiles/cachefiles.ko 2024-01-16T14:48:50.599 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ceph 2024-01-16T14:48:50.600 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ceph/ceph.ko 2024-01-16T14:48:50.600 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/coda 2024-01-16T14:48:50.600 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/coda/coda.ko 2024-01-16T14:48:50.600 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/cramfs 2024-01-16T14:48:50.600 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/cramfs/cramfs.ko 2024-01-16T14:48:50.600 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/dlm 2024-01-16T14:48:50.600 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/dlm/dlm.ko 2024-01-16T14:48:50.600 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ecryptfs 2024-01-16T14:48:50.600 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ecryptfs/ecryptfs.ko 2024-01-16T14:48:50.600 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/f2fs 2024-01-16T14:48:50.600 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/f2fs/f2fs.ko 2024-01-16T14:48:50.600 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/fat 2024-01-16T14:48:50.601 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/fat/fat.ko 2024-01-16T14:48:50.601 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/fat/msdos.ko 2024-01-16T14:48:50.601 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/fat/vfat.ko 2024-01-16T14:48:50.601 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/fscache 2024-01-16T14:48:50.601 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/fscache/fscache.ko 2024-01-16T14:48:50.601 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/fuse 2024-01-16T14:48:50.601 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/fuse/cuse.ko 2024-01-16T14:48:50.601 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/fuse/fuse.ko 2024-01-16T14:48:50.601 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/gfs2 2024-01-16T14:48:50.601 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/gfs2/gfs2.ko 2024-01-16T14:48:50.601 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/hfs 2024-01-16T14:48:50.601 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/hfs/hfs.ko 2024-01-16T14:48:50.601 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/hfsplus 2024-01-16T14:48:50.601 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/hfsplus/hfsplus.ko 2024-01-16T14:48:50.602 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/isofs 2024-01-16T14:48:50.602 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/isofs/isofs.ko 2024-01-16T14:48:50.602 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/jfs 2024-01-16T14:48:50.602 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/jfs/jfs.ko 2024-01-16T14:48:50.602 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/lockd 2024-01-16T14:48:50.602 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/lockd/lockd.ko 2024-01-16T14:48:50.602 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/minix 2024-01-16T14:48:50.602 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/minix/minix.ko 2024-01-16T14:48:50.602 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/netfs 2024-01-16T14:48:50.602 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/netfs/netfs.ko 2024-01-16T14:48:50.602 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs 2024-01-16T14:48:50.602 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/blocklayout 2024-01-16T14:48:50.602 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-01-16T14:48:50.602 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/filelayout 2024-01-16T14:48:50.603 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-01-16T14:48:50.603 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/flexfilelayout 2024-01-16T14:48:50.603 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-01-16T14:48:50.603 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/nfs.ko 2024-01-16T14:48:50.603 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/nfsv3.ko 2024-01-16T14:48:50.603 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/nfsv4.ko 2024-01-16T14:48:50.603 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs_common 2024-01-16T14:48:50.603 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs_common/grace.ko 2024-01-16T14:48:50.603 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs_common/nfs_acl.ko 2024-01-16T14:48:50.603 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfsd 2024-01-16T14:48:50.603 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfsd/nfsd.ko 2024-01-16T14:48:50.603 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nilfs2 2024-01-16T14:48:50.603 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nilfs2/nilfs2.ko 2024-01-16T14:48:50.603 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls 2024-01-16T14:48:50.603 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-celtic.ko 2024-01-16T14:48:50.604 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-centeuro.ko 2024-01-16T14:48:50.604 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-croatian.ko 2024-01-16T14:48:50.604 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-cyrillic.ko 2024-01-16T14:48:50.604 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-gaelic.ko 2024-01-16T14:48:50.604 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-greek.ko 2024-01-16T14:48:50.604 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-iceland.ko 2024-01-16T14:48:50.604 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-inuit.ko 2024-01-16T14:48:50.604 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-roman.ko 2024-01-16T14:48:50.604 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-romanian.ko 2024-01-16T14:48:50.604 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-turkish.ko 2024-01-16T14:48:50.604 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp1250.ko 2024-01-16T14:48:50.604 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp1251.ko 2024-01-16T14:48:50.604 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp1255.ko 2024-01-16T14:48:50.604 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp737.ko 2024-01-16T14:48:50.605 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp775.ko 2024-01-16T14:48:50.605 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp850.ko 2024-01-16T14:48:50.605 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp852.ko 2024-01-16T14:48:50.605 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp855.ko 2024-01-16T14:48:50.605 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp857.ko 2024-01-16T14:48:50.605 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp860.ko 2024-01-16T14:48:50.605 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp861.ko 2024-01-16T14:48:50.605 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp862.ko 2024-01-16T14:48:50.605 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp863.ko 2024-01-16T14:48:50.605 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp864.ko 2024-01-16T14:48:50.605 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp865.ko 2024-01-16T14:48:50.605 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp866.ko 2024-01-16T14:48:50.605 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp869.ko 2024-01-16T14:48:50.605 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp874.ko 2024-01-16T14:48:50.606 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp932.ko 2024-01-16T14:48:50.606 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp936.ko 2024-01-16T14:48:50.606 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp949.ko 2024-01-16T14:48:50.606 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp950.ko 2024-01-16T14:48:50.606 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_euc-jp.ko 2024-01-16T14:48:50.606 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-1.ko 2024-01-16T14:48:50.606 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-13.ko 2024-01-16T14:48:50.606 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-14.ko 2024-01-16T14:48:50.606 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-15.ko 2024-01-16T14:48:50.606 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-2.ko 2024-01-16T14:48:50.606 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-3.ko 2024-01-16T14:48:50.606 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-4.ko 2024-01-16T14:48:50.606 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-5.ko 2024-01-16T14:48:50.606 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-6.ko 2024-01-16T14:48:50.607 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-7.ko 2024-01-16T14:48:50.607 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-9.ko 2024-01-16T14:48:50.607 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_koi8-r.ko 2024-01-16T14:48:50.607 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_koi8-ru.ko 2024-01-16T14:48:50.607 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_koi8-u.ko 2024-01-16T14:48:50.607 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_ucs2_utils.ko 2024-01-16T14:48:50.607 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_utf8.ko 2024-01-16T14:48:50.607 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2 2024-01-16T14:48:50.607 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/cluster 2024-01-16T14:48:50.607 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-01-16T14:48:50.607 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/dlm 2024-01-16T14:48:50.607 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-01-16T14:48:50.607 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/dlmfs 2024-01-16T14:48:50.608 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-01-16T14:48:50.608 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/ocfs2.ko 2024-01-16T14:48:50.608 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-01-16T14:48:50.608 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-01-16T14:48:50.608 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-01-16T14:48:50.608 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/overlayfs 2024-01-16T14:48:50.608 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/overlayfs/overlay.ko 2024-01-16T14:48:50.608 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/pstore 2024-01-16T14:48:50.608 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/pstore/ramoops.ko 2024-01-16T14:48:50.608 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/reiserfs 2024-01-16T14:48:50.608 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/reiserfs/reiserfs.ko 2024-01-16T14:48:50.608 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/romfs 2024-01-16T14:48:50.608 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/romfs/romfs.ko 2024-01-16T14:48:50.608 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/smb 2024-01-16T14:48:50.609 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/smb/client 2024-01-16T14:48:50.609 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/smb/client/cifs.ko 2024-01-16T14:48:50.609 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/smb/common 2024-01-16T14:48:50.609 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/smb/common/cifs_arc4.ko 2024-01-16T14:48:50.609 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/smb/common/cifs_md4.ko 2024-01-16T14:48:50.609 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/squashfs 2024-01-16T14:48:50.609 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/squashfs/squashfs.ko 2024-01-16T14:48:50.609 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/sysv 2024-01-16T14:48:50.609 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/sysv/sysv.ko 2024-01-16T14:48:50.609 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/udf 2024-01-16T14:48:50.609 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/udf/udf.ko 2024-01-16T14:48:50.609 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ufs 2024-01-16T14:48:50.609 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ufs/ufs.ko 2024-01-16T14:48:50.610 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/xfs 2024-01-16T14:48:50.610 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/xfs/xfs.ko 2024-01-16T14:48:50.610 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/kernel 2024-01-16T14:48:50.610 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/kernel/rcu 2024-01-16T14:48:50.610 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/kernel/rcu/rcutorture.ko 2024-01-16T14:48:50.610 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/kernel/torture.ko 2024-01-16T14:48:50.610 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/kernel/trace 2024-01-16T14:48:50.610 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-01-16T14:48:50.610 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib 2024-01-16T14:48:50.610 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/asn1_encoder.ko 2024-01-16T14:48:50.610 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/crc-itu-t.ko 2024-01-16T14:48:50.610 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/crc8.ko 2024-01-16T14:48:50.610 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/crypto 2024-01-16T14:48:50.610 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/crypto/libarc4.ko 2024-01-16T14:48:50.611 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/crypto/libdes.ko 2024-01-16T14:48:50.611 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/libcrc32c.ko 2024-01-16T14:48:50.611 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/lru_cache.ko 2024-01-16T14:48:50.611 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/lz4 2024-01-16T14:48:50.611 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/lz4/lz4_compress.ko 2024-01-16T14:48:50.611 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/lz4/lz4hc_compress.ko 2024-01-16T14:48:50.611 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/math 2024-01-16T14:48:50.611 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/math/cordic.ko 2024-01-16T14:48:50.611 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/raid6 2024-01-16T14:48:50.611 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/raid6/raid6_pq.ko 2024-01-16T14:48:50.611 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/reed_solomon 2024-01-16T14:48:50.611 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/reed_solomon/reed_solomon.ko 2024-01-16T14:48:50.611 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/ts_bm.ko 2024-01-16T14:48:50.611 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/ts_fsm.ko 2024-01-16T14:48:50.611 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/ts_kmp.ko 2024-01-16T14:48:50.612 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/zstd 2024-01-16T14:48:50.612 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/zstd/zstd_compress.ko 2024-01-16T14:48:50.612 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/mm 2024-01-16T14:48:50.612 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/mm/hwpoison-inject.ko 2024-01-16T14:48:50.612 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net 2024-01-16T14:48:50.612 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan 2024-01-16T14:48:50.612 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/6lowpan.ko 2024-01-16T14:48:50.612 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_dest.ko 2024-01-16T14:48:50.612 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_fragment.ko 2024-01-16T14:48:50.612 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_hop.ko 2024-01-16T14:48:50.612 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_ipv6.ko 2024-01-16T14:48:50.612 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_mobility.ko 2024-01-16T14:48:50.612 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_routing.ko 2024-01-16T14:48:50.612 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_udp.ko 2024-01-16T14:48:50.613 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/802 2024-01-16T14:48:50.613 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/802/garp.ko 2024-01-16T14:48:50.613 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/802/mrp.ko 2024-01-16T14:48:50.613 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/802/p8022.ko 2024-01-16T14:48:50.613 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/802/psnap.ko 2024-01-16T14:48:50.613 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/802/stp.ko 2024-01-16T14:48:50.613 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/8021q 2024-01-16T14:48:50.613 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/8021q/8021q.ko 2024-01-16T14:48:50.613 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/9p 2024-01-16T14:48:50.613 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/9p/9pnet.ko 2024-01-16T14:48:50.613 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/9p/9pnet_fd.ko 2024-01-16T14:48:50.613 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/9p/9pnet_rdma.ko 2024-01-16T14:48:50.613 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/9p/9pnet_virtio.ko 2024-01-16T14:48:50.613 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/batman-adv 2024-01-16T14:48:50.613 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/batman-adv/batman-adv.ko 2024-01-16T14:48:50.614 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge 2024-01-16T14:48:50.614 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/br_netfilter.ko 2024-01-16T14:48:50.614 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/bridge.ko 2024-01-16T14:48:50.614 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter 2024-01-16T14:48:50.614 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-01-16T14:48:50.614 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_among.ko 2024-01-16T14:48:50.614 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_arp.ko 2024-01-16T14:48:50.614 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-01-16T14:48:50.614 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-01-16T14:48:50.614 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_ip.ko 2024-01-16T14:48:50.614 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-01-16T14:48:50.614 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_limit.ko 2024-01-16T14:48:50.614 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_log.ko 2024-01-16T14:48:50.614 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_mark.ko 2024-01-16T14:48:50.615 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-01-16T14:48:50.615 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-01-16T14:48:50.615 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-01-16T14:48:50.615 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-01-16T14:48:50.615 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_snat.ko 2024-01-16T14:48:50.615 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_stp.ko 2024-01-16T14:48:50.615 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-01-16T14:48:50.615 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-01-16T14:48:50.615 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-01-16T14:48:50.615 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-01-16T14:48:50.615 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebtables.ko 2024-01-16T14:48:50.615 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-01-16T14:48:50.615 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-01-16T14:48:50.615 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-01-16T14:48:50.615 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ceph 2024-01-16T14:48:50.616 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ceph/libceph.ko 2024-01-16T14:48:50.616 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/core 2024-01-16T14:48:50.616 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/core/failover.ko 2024-01-16T14:48:50.616 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/core/pktgen.ko 2024-01-16T14:48:50.616 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dccp 2024-01-16T14:48:50.616 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dccp/dccp.ko 2024-01-16T14:48:50.616 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dccp/dccp_diag.ko 2024-01-16T14:48:50.616 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dccp/dccp_ipv4.ko 2024-01-16T14:48:50.616 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dccp/dccp_ipv6.ko 2024-01-16T14:48:50.616 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dns_resolver 2024-01-16T14:48:50.616 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dns_resolver/dns_resolver.ko 2024-01-16T14:48:50.616 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dsa 2024-01-16T14:48:50.616 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dsa/dsa_core.ko 2024-01-16T14:48:50.616 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dsa/tag_brcm.ko 2024-01-16T14:48:50.617 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dsa/tag_dsa.ko 2024-01-16T14:48:50.617 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dsa/tag_none.ko 2024-01-16T14:48:50.617 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dsa/tag_trailer.ko 2024-01-16T14:48:50.617 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ieee802154 2024-01-16T14:48:50.617 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ieee802154/6lowpan 2024-01-16T14:48:50.617 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-01-16T14:48:50.617 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ieee802154/ieee802154.ko 2024-01-16T14:48:50.617 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ieee802154/ieee802154_socket.ko 2024-01-16T14:48:50.617 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4 2024-01-16T14:48:50.617 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/ah4.ko 2024-01-16T14:48:50.617 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/esp4.ko 2024-01-16T14:48:50.617 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/fou.ko 2024-01-16T14:48:50.617 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/gre.ko 2024-01-16T14:48:50.617 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/inet_diag.ko 2024-01-16T14:48:50.617 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/ip_gre.ko 2024-01-16T14:48:50.618 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/ip_tunnel.ko 2024-01-16T14:48:50.618 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/ip_vti.ko 2024-01-16T14:48:50.618 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/ipcomp.ko 2024-01-16T14:48:50.618 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/ipip.ko 2024-01-16T14:48:50.618 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter 2024-01-16T14:48:50.618 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/arp_tables.ko 2024-01-16T14:48:50.618 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-01-16T14:48:50.618 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-01-16T14:48:50.618 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-01-16T14:48:50.618 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-01-16T14:48:50.618 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-01-16T14:48:50.618 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-01-16T14:48:50.618 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-01-16T14:48:50.618 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-01-16T14:48:50.619 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-01-16T14:48:50.619 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/iptable_security.ko 2024-01-16T14:48:50.619 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-01-16T14:48:50.619 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-01-16T14:48:50.619 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-01-16T14:48:50.619 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-01-16T14:48:50.619 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-01-16T14:48:50.619 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-01-16T14:48:50.619 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-01-16T14:48:50.619 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-01-16T14:48:50.619 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-01-16T14:48:50.619 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-01-16T14:48:50.619 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_bic.ko 2024-01-16T14:48:50.619 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_dctcp.ko 2024-01-16T14:48:50.619 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_diag.ko 2024-01-16T14:48:50.620 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_highspeed.ko 2024-01-16T14:48:50.620 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_htcp.ko 2024-01-16T14:48:50.620 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_hybla.ko 2024-01-16T14:48:50.620 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_illinois.ko 2024-01-16T14:48:50.620 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_lp.ko 2024-01-16T14:48:50.620 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_scalable.ko 2024-01-16T14:48:50.620 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_vegas.ko 2024-01-16T14:48:50.620 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_veno.ko 2024-01-16T14:48:50.620 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_westwood.ko 2024-01-16T14:48:50.620 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_yeah.ko 2024-01-16T14:48:50.620 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tunnel4.ko 2024-01-16T14:48:50.620 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/udp_diag.ko 2024-01-16T14:48:50.620 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/udp_tunnel.ko 2024-01-16T14:48:50.621 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/xfrm4_tunnel.ko 2024-01-16T14:48:50.621 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6 2024-01-16T14:48:50.621 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/ah6.ko 2024-01-16T14:48:50.621 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/esp6.ko 2024-01-16T14:48:50.621 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/fou6.ko 2024-01-16T14:48:50.621 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/ip6_tunnel.ko 2024-01-16T14:48:50.621 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-01-16T14:48:50.621 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/ip6_vti.ko 2024-01-16T14:48:50.621 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/ipcomp6.ko 2024-01-16T14:48:50.621 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter 2024-01-16T14:48:50.621 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-01-16T14:48:50.621 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-01-16T14:48:50.621 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-01-16T14:48:50.621 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-01-16T14:48:50.621 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-01-16T14:48:50.622 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-01-16T14:48:50.622 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-01-16T14:48:50.622 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-01-16T14:48:50.622 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-01-16T14:48:50.622 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-01-16T14:48:50.622 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-01-16T14:48:50.622 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-01-16T14:48:50.622 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-01-16T14:48:50.622 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-01-16T14:48:50.622 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-01-16T14:48:50.622 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-01-16T14:48:50.622 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-01-16T14:48:50.622 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-01-16T14:48:50.622 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-01-16T14:48:50.623 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-01-16T14:48:50.623 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-01-16T14:48:50.623 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-01-16T14:48:50.623 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-01-16T14:48:50.623 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-01-16T14:48:50.623 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/sit.ko 2024-01-16T14:48:50.623 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/tunnel6.ko 2024-01-16T14:48:50.623 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/xfrm6_tunnel.ko 2024-01-16T14:48:50.623 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/key 2024-01-16T14:48:50.623 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/key/af_key.ko 2024-01-16T14:48:50.623 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp 2024-01-16T14:48:50.623 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_core.ko 2024-01-16T14:48:50.623 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_debugfs.ko 2024-01-16T14:48:50.623 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_eth.ko 2024-01-16T14:48:50.624 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_ip.ko 2024-01-16T14:48:50.624 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_ip6.ko 2024-01-16T14:48:50.624 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_netlink.ko 2024-01-16T14:48:50.624 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_ppp.ko 2024-01-16T14:48:50.624 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/llc 2024-01-16T14:48:50.624 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/llc/llc.ko 2024-01-16T14:48:50.624 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/mac802154 2024-01-16T14:48:50.624 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/mac802154/mac802154.ko 2024-01-16T14:48:50.624 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/mpls 2024-01-16T14:48:50.624 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/mpls/mpls_gso.ko 2024-01-16T14:48:50.624 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter 2024-01-16T14:48:50.624 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset 2024-01-16T14:48:50.624 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set.ko 2024-01-16T14:48:50.625 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-01-16T14:48:50.625 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-01-16T14:48:50.625 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-01-16T14:48:50.625 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-01-16T14:48:50.625 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-01-16T14:48:50.625 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-01-16T14:48:50.625 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-01-16T14:48:50.625 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-01-16T14:48:50.625 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-01-16T14:48:50.625 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-01-16T14:48:50.625 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-01-16T14:48:50.625 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-01-16T14:48:50.625 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-01-16T14:48:50.625 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-01-16T14:48:50.625 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-01-16T14:48:50.626 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs 2024-01-16T14:48:50.626 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs.ko 2024-01-16T14:48:50.626 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-01-16T14:48:50.626 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-01-16T14:48:50.626 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-01-16T14:48:50.626 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-01-16T14:48:50.626 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-01-16T14:48:50.626 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-01-16T14:48:50.626 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-01-16T14:48:50.626 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-01-16T14:48:50.626 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-01-16T14:48:50.626 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-01-16T14:48:50.626 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-01-16T14:48:50.626 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-01-16T14:48:50.627 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-01-16T14:48:50.627 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conncount.ko 2024-01-16T14:48:50.627 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack.ko 2024-01-16T14:48:50.627 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-01-16T14:48:50.627 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-01-16T14:48:50.627 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-01-16T14:48:50.627 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_h323.ko 2024-01-16T14:48:50.627 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_irc.ko 2024-01-16T14:48:50.627 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-01-16T14:48:50.627 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-01-16T14:48:50.627 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-01-16T14:48:50.627 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_sane.ko 2024-01-16T14:48:50.627 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_sip.ko 2024-01-16T14:48:50.627 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-01-16T14:48:50.627 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-01-16T14:48:50.628 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_dup_netdev.ko 2024-01-16T14:48:50.628 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_flow_table.ko 2024-01-16T14:48:50.628 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_flow_table_inet.ko 2024-01-16T14:48:50.628 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_log_syslog.ko 2024-01-16T14:48:50.628 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_nat.ko 2024-01-16T14:48:50.628 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_nat_amanda.ko 2024-01-16T14:48:50.628 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_nat_ftp.ko 2024-01-16T14:48:50.628 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_nat_irc.ko 2024-01-16T14:48:50.628 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_nat_sip.ko 2024-01-16T14:48:50.628 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_nat_tftp.ko 2024-01-16T14:48:50.628 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_synproxy_core.ko 2024-01-16T14:48:50.628 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_tables.ko 2024-01-16T14:48:50.628 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nfnetlink.ko 2024-01-16T14:48:50.629 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nfnetlink_acct.ko 2024-01-16T14:48:50.629 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nfnetlink_log.ko 2024-01-16T14:48:50.629 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nfnetlink_osf.ko 2024-01-16T14:48:50.629 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nfnetlink_queue.ko 2024-01-16T14:48:50.629 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_chain_nat.ko 2024-01-16T14:48:50.629 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_compat.ko 2024-01-16T14:48:50.629 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_ct.ko 2024-01-16T14:48:50.629 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_dup_netdev.ko 2024-01-16T14:48:50.629 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_fib.ko 2024-01-16T14:48:50.629 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_fwd_netdev.ko 2024-01-16T14:48:50.629 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_hash.ko 2024-01-16T14:48:50.629 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_limit.ko 2024-01-16T14:48:50.629 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_log.ko 2024-01-16T14:48:50.629 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_masq.ko 2024-01-16T14:48:50.630 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_nat.ko 2024-01-16T14:48:50.630 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_numgen.ko 2024-01-16T14:48:50.630 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_queue.ko 2024-01-16T14:48:50.630 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_quota.ko 2024-01-16T14:48:50.630 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_redir.ko 2024-01-16T14:48:50.630 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_reject.ko 2024-01-16T14:48:50.630 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_reject_inet.ko 2024-01-16T14:48:50.630 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_socket.ko 2024-01-16T14:48:50.630 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_tunnel.ko 2024-01-16T14:48:50.630 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_AUDIT.ko 2024-01-16T14:48:50.630 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_CHECKSUM.ko 2024-01-16T14:48:50.630 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_CLASSIFY.ko 2024-01-16T14:48:50.630 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-01-16T14:48:50.630 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_CT.ko 2024-01-16T14:48:50.631 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_DSCP.ko 2024-01-16T14:48:50.631 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_HL.ko 2024-01-16T14:48:50.631 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_HMARK.ko 2024-01-16T14:48:50.631 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_IDLETIMER.ko 2024-01-16T14:48:50.631 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_LED.ko 2024-01-16T14:48:50.631 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_LOG.ko 2024-01-16T14:48:50.631 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_MASQUERADE.ko 2024-01-16T14:48:50.631 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_NETMAP.ko 2024-01-16T14:48:50.631 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_NFLOG.ko 2024-01-16T14:48:50.631 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_NFQUEUE.ko 2024-01-16T14:48:50.631 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_RATEEST.ko 2024-01-16T14:48:50.631 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_REDIRECT.ko 2024-01-16T14:48:50.631 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_SECMARK.ko 2024-01-16T14:48:50.631 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_TCPMSS.ko 2024-01-16T14:48:50.631 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-01-16T14:48:50.632 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_TEE.ko 2024-01-16T14:48:50.632 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_TPROXY.ko 2024-01-16T14:48:50.632 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_TRACE.ko 2024-01-16T14:48:50.632 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_addrtype.ko 2024-01-16T14:48:50.632 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_bpf.ko 2024-01-16T14:48:50.632 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_cgroup.ko 2024-01-16T14:48:50.632 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_cluster.ko 2024-01-16T14:48:50.632 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_comment.ko 2024-01-16T14:48:50.632 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_connbytes.ko 2024-01-16T14:48:50.632 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_connlabel.ko 2024-01-16T14:48:50.632 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_connlimit.ko 2024-01-16T14:48:50.632 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_connmark.ko 2024-01-16T14:48:50.632 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_conntrack.ko 2024-01-16T14:48:50.632 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_cpu.ko 2024-01-16T14:48:50.633 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_dccp.ko 2024-01-16T14:48:50.633 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_devgroup.ko 2024-01-16T14:48:50.633 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_dscp.ko 2024-01-16T14:48:50.633 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_ecn.ko 2024-01-16T14:48:50.633 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_esp.ko 2024-01-16T14:48:50.633 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_hashlimit.ko 2024-01-16T14:48:50.633 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_helper.ko 2024-01-16T14:48:50.633 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_hl.ko 2024-01-16T14:48:50.633 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_ipcomp.ko 2024-01-16T14:48:50.633 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_iprange.ko 2024-01-16T14:48:50.633 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_ipvs.ko 2024-01-16T14:48:50.633 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_l2tp.ko 2024-01-16T14:48:50.633 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_length.ko 2024-01-16T14:48:50.633 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_limit.ko 2024-01-16T14:48:50.633 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_mac.ko 2024-01-16T14:48:50.634 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_mark.ko 2024-01-16T14:48:50.634 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_multiport.ko 2024-01-16T14:48:50.634 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_nat.ko 2024-01-16T14:48:50.634 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_nfacct.ko 2024-01-16T14:48:50.634 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_osf.ko 2024-01-16T14:48:50.634 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_owner.ko 2024-01-16T14:48:50.634 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_physdev.ko 2024-01-16T14:48:50.634 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_pkttype.ko 2024-01-16T14:48:50.634 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_policy.ko 2024-01-16T14:48:50.634 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_quota.ko 2024-01-16T14:48:50.634 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_rateest.ko 2024-01-16T14:48:50.634 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_realm.ko 2024-01-16T14:48:50.634 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_recent.ko 2024-01-16T14:48:50.634 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_sctp.ko 2024-01-16T14:48:50.635 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_set.ko 2024-01-16T14:48:50.635 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_socket.ko 2024-01-16T14:48:50.635 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_state.ko 2024-01-16T14:48:50.635 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_statistic.ko 2024-01-16T14:48:50.635 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_string.ko 2024-01-16T14:48:50.635 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_tcpmss.ko 2024-01-16T14:48:50.635 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_time.ko 2024-01-16T14:48:50.635 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_u32.ko 2024-01-16T14:48:50.635 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netlink 2024-01-16T14:48:50.635 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netlink/netlink_diag.ko 2024-01-16T14:48:50.635 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/nsh 2024-01-16T14:48:50.635 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/nsh/nsh.ko 2024-01-16T14:48:50.635 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/openvswitch 2024-01-16T14:48:50.635 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/openvswitch/openvswitch.ko 2024-01-16T14:48:50.635 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/openvswitch/vport-geneve.ko 2024-01-16T14:48:50.636 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/openvswitch/vport-gre.ko 2024-01-16T14:48:50.636 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/openvswitch/vport-vxlan.ko 2024-01-16T14:48:50.636 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/packet 2024-01-16T14:48:50.636 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/packet/af_packet_diag.ko 2024-01-16T14:48:50.636 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/rds 2024-01-16T14:48:50.636 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/rds/rds.ko 2024-01-16T14:48:50.636 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/rds/rds_rdma.ko 2024-01-16T14:48:50.636 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/rds/rds_tcp.ko 2024-01-16T14:48:50.636 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched 2024-01-16T14:48:50.636 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_csum.ko 2024-01-16T14:48:50.636 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_gact.ko 2024-01-16T14:48:50.636 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_ipt.ko 2024-01-16T14:48:50.636 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_mirred.ko 2024-01-16T14:48:50.636 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_nat.ko 2024-01-16T14:48:50.637 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_pedit.ko 2024-01-16T14:48:50.637 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_police.ko 2024-01-16T14:48:50.637 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_simple.ko 2024-01-16T14:48:50.637 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_skbedit.ko 2024-01-16T14:48:50.637 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_vlan.ko 2024-01-16T14:48:50.637 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/cls_basic.ko 2024-01-16T14:48:50.637 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/cls_bpf.ko 2024-01-16T14:48:50.637 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/cls_flow.ko 2024-01-16T14:48:50.637 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/cls_fw.ko 2024-01-16T14:48:50.637 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/cls_route.ko 2024-01-16T14:48:50.637 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/cls_u32.ko 2024-01-16T14:48:50.637 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/em_cmp.ko 2024-01-16T14:48:50.637 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/em_ipset.ko 2024-01-16T14:48:50.637 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/em_meta.ko 2024-01-16T14:48:50.637 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/em_nbyte.ko 2024-01-16T14:48:50.638 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/em_text.ko 2024-01-16T14:48:50.638 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/em_u32.ko 2024-01-16T14:48:50.638 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_choke.ko 2024-01-16T14:48:50.638 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_codel.ko 2024-01-16T14:48:50.638 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_drr.ko 2024-01-16T14:48:50.638 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_fq.ko 2024-01-16T14:48:50.638 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_gred.ko 2024-01-16T14:48:50.638 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_hfsc.ko 2024-01-16T14:48:50.638 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_hhf.ko 2024-01-16T14:48:50.638 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_htb.ko 2024-01-16T14:48:50.638 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_ingress.ko 2024-01-16T14:48:50.638 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_mqprio.ko 2024-01-16T14:48:50.638 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_mqprio_lib.ko 2024-01-16T14:48:50.638 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_multiq.ko 2024-01-16T14:48:50.639 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_netem.ko 2024-01-16T14:48:50.639 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_pie.ko 2024-01-16T14:48:50.639 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_plug.ko 2024-01-16T14:48:50.639 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_prio.ko 2024-01-16T14:48:50.639 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_qfq.ko 2024-01-16T14:48:50.639 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_red.ko 2024-01-16T14:48:50.639 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_sfb.ko 2024-01-16T14:48:50.639 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_sfq.ko 2024-01-16T14:48:50.639 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_tbf.ko 2024-01-16T14:48:50.639 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_teql.ko 2024-01-16T14:48:50.639 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sctp 2024-01-16T14:48:50.639 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sctp/sctp.ko 2024-01-16T14:48:50.639 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sctp/sctp_diag.ko 2024-01-16T14:48:50.639 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc 2024-01-16T14:48:50.640 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc/auth_gss 2024-01-16T14:48:50.640 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-01-16T14:48:50.640 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-01-16T14:48:50.640 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc/sunrpc.ko 2024-01-16T14:48:50.640 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc/xprtrdma 2024-01-16T14:48:50.640 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-01-16T14:48:50.640 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/tipc 2024-01-16T14:48:50.640 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/tipc/diag.ko 2024-01-16T14:48:50.640 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/tipc/tipc.ko 2024-01-16T14:48:50.640 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/unix 2024-01-16T14:48:50.640 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/unix/unix_diag.ko 2024-01-16T14:48:50.640 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/vmw_vsock 2024-01-16T14:48:50.640 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-01-16T14:48:50.640 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-01-16T14:48:50.641 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/vmw_vsock/vsock.ko 2024-01-16T14:48:50.641 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/vmw_vsock/vsock_diag.ko 2024-01-16T14:48:50.641 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/vmw_vsock/vsock_loopback.ko 2024-01-16T14:48:50.641 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/xfrm 2024-01-16T14:48:50.641 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/xfrm/xfrm_ipcomp.ko 2024-01-16T14:48:50.641 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/security 2024-01-16T14:48:50.641 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/security/keys 2024-01-16T14:48:50.641 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/security/keys/encrypted-keys 2024-01-16T14:48:50.641 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-01-16T14:48:50.641 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/security/keys/trusted-keys 2024-01-16T14:48:50.641 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/security/keys/trusted-keys/trusted.ko 2024-01-16T14:48:50.641 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/virt 2024-01-16T14:48:50.641 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/virt/lib 2024-01-16T14:48:50.641 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/virt/lib/irqbypass.ko 2024-01-16T14:48:50.641 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/modules.alias 2024-01-16T14:48:50.642 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/modules.alias.bin 2024-01-16T14:48:50.642 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/modules.builtin 2024-01-16T14:48:50.642 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/modules.builtin.alias.bin 2024-01-16T14:48:50.642 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/modules.builtin.bin 2024-01-16T14:48:50.642 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/modules.builtin.modinfo 2024-01-16T14:48:50.642 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/modules.dep 2024-01-16T14:48:50.642 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/modules.dep.bin 2024-01-16T14:48:50.642 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/modules.devname 2024-01-16T14:48:50.642 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/modules.order 2024-01-16T14:48:50.642 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/modules.softdep 2024-01-16T14:48:50.642 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/modules.symbols 2024-01-16T14:48:50.642 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.7.0-g2668949e0971/modules.symbols.bin 2024-01-16T14:48:50.643 DEBUG:teuthology.task.kernel:get_image_version: 6.7.0-g2668949e0971 2024-01-16T14:48:50.643 DEBUG:teuthology.orchestra.run.smithi204:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-01-16T14:48:50.665 INFO:teuthology.orchestra.run.smithi204.stdout:postinstall scriptlet (using /bin/sh): 2024-01-16T14:48:50.665 INFO:teuthology.orchestra.run.smithi204.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.7.0-g2668949e0971 -a -r /boot/System.map-6.7.0-g2668949e0971 ]; then 2024-01-16T14:48:50.665 INFO:teuthology.orchestra.run.smithi204.stdout:cp /boot/vmlinuz-6.7.0-g2668949e0971 /boot/.vmlinuz-6.7.0-g2668949e0971-rpm 2024-01-16T14:48:50.665 INFO:teuthology.orchestra.run.smithi204.stdout:cp /boot/System.map-6.7.0-g2668949e0971 /boot/.System.map-6.7.0-g2668949e0971-rpm 2024-01-16T14:48:50.666 INFO:teuthology.orchestra.run.smithi204.stdout:rm -f /boot/vmlinuz-6.7.0-g2668949e0971 /boot/System.map-6.7.0-g2668949e0971 2024-01-16T14:48:50.666 INFO:teuthology.orchestra.run.smithi204.stdout:/sbin/installkernel 6.7.0-g2668949e0971 /boot/.vmlinuz-6.7.0-g2668949e0971-rpm /boot/.System.map-6.7.0-g2668949e0971-rpm 2024-01-16T14:48:50.666 INFO:teuthology.orchestra.run.smithi204.stdout:rm -f /boot/.vmlinuz-6.7.0-g2668949e0971-rpm /boot/.System.map-6.7.0-g2668949e0971-rpm 2024-01-16T14:48:50.666 INFO:teuthology.orchestra.run.smithi204.stdout:fi 2024-01-16T14:48:50.666 INFO:teuthology.orchestra.run.smithi204.stdout:preuninstall scriptlet (using /bin/sh): 2024-01-16T14:48:50.666 INFO:teuthology.orchestra.run.smithi204.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-01-16T14:48:50.666 INFO:teuthology.orchestra.run.smithi204.stdout:new-kernel-pkg --remove 6.7.0-g2668949e0971 --rminitrd --initrdfile=/boot/initramfs-6.7.0-g2668949e0971.img 2024-01-16T14:48:50.666 INFO:teuthology.orchestra.run.smithi204.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-01-16T14:48:50.666 INFO:teuthology.orchestra.run.smithi204.stdout:kernel-install remove 6.7.0-g2668949e0971 2024-01-16T14:48:50.666 INFO:teuthology.orchestra.run.smithi204.stdout:fi 2024-01-16T14:48:50.666 INFO:teuthology.orchestra.run.smithi204.stdout:postuninstall scriptlet (using /bin/sh): 2024-01-16T14:48:50.666 INFO:teuthology.orchestra.run.smithi204.stdout:if [ -x /sbin/update-bootloader ]; then 2024-01-16T14:48:50.666 INFO:teuthology.orchestra.run.smithi204.stdout:/sbin/update-bootloader --remove 6.7.0-g2668949e0971 2024-01-16T14:48:50.666 INFO:teuthology.orchestra.run.smithi204.stdout:fi 2024-01-16T14:48:50.667 DEBUG:teuthology.orchestra.run.smithi204:> sudo rpm -qi grub2-tools 2024-01-16T14:48:50.747 INFO:teuthology.orchestra.run.smithi204.stdout:Name : grub2-tools 2024-01-16T14:48:50.747 INFO:teuthology.orchestra.run.smithi204.stdout:Epoch : 1 2024-01-16T14:48:50.747 INFO:teuthology.orchestra.run.smithi204.stdout:Version : 2.06 2024-01-16T14:48:50.748 INFO:teuthology.orchestra.run.smithi204.stdout:Release : 68.el9 2024-01-16T14:48:50.748 INFO:teuthology.orchestra.run.smithi204.stdout:Architecture: x86_64 2024-01-16T14:48:50.748 INFO:teuthology.orchestra.run.smithi204.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-01-16T14:48:50.748 INFO:teuthology.orchestra.run.smithi204.stdout:Group : Unspecified 2024-01-16T14:48:50.748 INFO:teuthology.orchestra.run.smithi204.stdout:Size : 8264254 2024-01-16T14:48:50.748 INFO:teuthology.orchestra.run.smithi204.stdout:License : GPLv3+ 2024-01-16T14:48:50.748 INFO:teuthology.orchestra.run.smithi204.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-01-16T14:48:50.748 INFO:teuthology.orchestra.run.smithi204.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-01-16T14:48:50.748 INFO:teuthology.orchestra.run.smithi204.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-01-16T14:48:50.748 INFO:teuthology.orchestra.run.smithi204.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-01-16T14:48:50.748 INFO:teuthology.orchestra.run.smithi204.stdout:Packager : builder@centos.org 2024-01-16T14:48:50.748 INFO:teuthology.orchestra.run.smithi204.stdout:Vendor : CentOS 2024-01-16T14:48:50.748 INFO:teuthology.orchestra.run.smithi204.stdout:URL : http://www.gnu.org/software/grub/ 2024-01-16T14:48:50.748 INFO:teuthology.orchestra.run.smithi204.stdout:Summary : Support tools for GRUB. 2024-01-16T14:48:50.749 INFO:teuthology.orchestra.run.smithi204.stdout:Description : 2024-01-16T14:48:50.749 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:48:50.749 INFO:teuthology.orchestra.run.smithi204.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-01-16T14:48:50.749 INFO:teuthology.orchestra.run.smithi204.stdout:customizable bootloader with modular architecture. It supports a rich 2024-01-16T14:48:50.749 INFO:teuthology.orchestra.run.smithi204.stdout:variety of kernel formats, file systems, computer architectures and 2024-01-16T14:48:50.749 INFO:teuthology.orchestra.run.smithi204.stdout:hardware devices. 2024-01-16T14:48:50.749 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:48:50.749 INFO:teuthology.orchestra.run.smithi204.stdout:This subpackage provides tools for support of all platforms. 2024-01-16T14:48:50.749 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-01-16T14:48:50.749 INFO:teuthology.task.kernel:Updating grub on smithi204 to boot 6.7.0-g2668949e0971 2024-01-16T14:48:50.749 DEBUG:teuthology.orchestra.run.smithi204:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-01-16T14:48:51.420 INFO:teuthology.orchestra.run.smithi204.stderr:Generating grub configuration file ... 2024-01-16T14:48:52.823 INFO:teuthology.orchestra.run.smithi204.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-01-16T14:48:52.860 INFO:teuthology.orchestra.run.smithi204.stderr:done 2024-01-16T14:48:52.863 DEBUG:teuthology.orchestra.run.smithi204:> mktemp 2024-01-16T14:48:52.878 INFO:teuthology.orchestra.run.smithi204.stdout:/tmp/tmp.5XFKChKbiA 2024-01-16T14:48:52.878 DEBUG:teuthology.orchestra.run.smithi204:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.5XFKChKbiA 2024-01-16T14:48:52.955 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 0666 /tmp/tmp.5XFKChKbiA 2024-01-16T14:48:53.099 DEBUG:teuthology.orchestra.remote:smithi204:/tmp/tmp.5XFKChKbiA is 6KB 2024-01-16T14:48:53.147 DEBUG:teuthology.orchestra.run.smithi204:> rm -fr /tmp/tmp.5XFKChKbiA 2024-01-16T14:48:53.161 DEBUG:teuthology.orchestra.run.smithi204:> sudo /bin/ls /boot/loader/entries || true 2024-01-16T14:48:53.228 INFO:teuthology.orchestra.run.smithi204.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-01-16T14:48:53.229 INFO:teuthology.orchestra.run.smithi204.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-01-16T14:48:53.229 INFO:teuthology.orchestra.run.smithi204.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-01-16T14:48:53.229 INFO:teuthology.orchestra.run.smithi204.stdout:15949e3c88704300882482eddfd7baae-6.7.0-g2668949e0971.conf 2024-01-16T14:48:53.230 DEBUG:teuthology.orchestra.run.smithi204:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.7.0-g2668949e0971 2024-01-16T14:48:53.387 DEBUG:teuthology.orchestra.run.smithi204:> sudo shutdown -r now 2024-01-16T14:49:05.192 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-01-16T14:49:05.192 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-01-16T14:49:05.193 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-16T14:49:23.408 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-01-16T14:49:23.409 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-01-16T14:49:23.410 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-16T14:49:23.516 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.143 2024-01-16T14:49:32.524 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-01-16T14:49:32.524 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-16T14:49:41.952 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.204 2024-01-16T14:49:50.962 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-01-16T14:49:50.963 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-16T14:49:54.048 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.204 2024-01-16T14:50:06.054 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-01-16T14:50:06.055 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-16T14:50:24.444 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.204 2024-01-16T14:50:32.532 DEBUG:teuthology.orchestra.remote:timed out 2024-01-16T14:50:39.452 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-01-16T14:50:39.453 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-16T14:50:42.524 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.204 2024-01-16T14:50:44.534 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-01-16T14:50:44.535 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-16T14:50:44.991 DEBUG:teuthology.orchestra.run.smithi143:> true 2024-01-16T14:50:45.655 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-01-16T14:50:45.655 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.7.0-g2668949e0971"... 2024-01-16T14:50:45.655 DEBUG:teuthology.orchestra.run.smithi143:> uname -r 2024-01-16T14:50:45.707 INFO:teuthology.orchestra.run.smithi143.stdout:6.7.0-g2668949e0971 2024-01-16T14:50:45.708 DEBUG:teuthology.task.kernel:current kernel version is 6.7.0-g2668949e0971 vs 6.7.0-g2668949e0971 2024-01-16T14:50:45.708 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-01-16T14:50:45.708 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-01-16T14:50:46.709 DEBUG:teuthology.parallel:result is None 2024-01-16T14:51:00.525 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-01-16T14:51:00.526 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-16T14:51:00.922 DEBUG:teuthology.orchestra.run.smithi204:> true 2024-01-16T14:51:01.602 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-01-16T14:51:01.602 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.7.0-g2668949e0971"... 2024-01-16T14:51:01.602 DEBUG:teuthology.orchestra.run.smithi204:> uname -r 2024-01-16T14:51:01.646 INFO:teuthology.orchestra.run.smithi204.stdout:6.7.0-g2668949e0971 2024-01-16T14:51:01.646 DEBUG:teuthology.task.kernel:current kernel version is 6.7.0-g2668949e0971 vs 6.7.0-g2668949e0971 2024-01-16T14:51:01.646 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-01-16T14:51:01.646 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-01-16T14:51:02.648 DEBUG:teuthology.parallel:result is None 2024-01-16T14:51:02.648 INFO:teuthology.run_tasks:Running task internal.base... 2024-01-16T14:51:02.662 INFO:teuthology.task.internal:Creating test directory... 2024-01-16T14:51:02.662 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-01-16T14:51:02.670 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-01-16T14:51:02.725 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-01-16T14:51:02.732 INFO:teuthology.run_tasks:Running task internal.archive... 2024-01-16T14:51:02.738 INFO:teuthology.task.internal:Creating archive directory... 2024-01-16T14:51:02.738 DEBUG:teuthology.orchestra.run.smithi143:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-01-16T14:51:02.746 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-01-16T14:51:02.889 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-01-16T14:51:02.896 INFO:teuthology.task.internal:Enabling coredump saving... 2024-01-16T14:51:02.896 DEBUG:teuthology.orchestra.run.smithi143:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-01-16T14:51:02.904 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-01-16T14:51:03.056 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-16T14:51:03.125 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-16T14:51:03.126 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-16T14:51:03.187 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-16T14:51:03.191 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-01-16T14:51:03.198 INFO:teuthology.task.internal:Configuring sudo... 2024-01-16T14:51:03.198 DEBUG:teuthology.orchestra.run.smithi143:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-01-16T14:51:03.206 DEBUG:teuthology.orchestra.run.smithi204:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-01-16T14:51:03.297 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-01-16T14:51:03.306 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-01-16T14:51:03.306 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-01-16T14:51:03.314 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-01-16T14:51:03.366 DEBUG:teuthology.orchestra.run.smithi143:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-16T14:51:03.447 DEBUG:teuthology.orchestra.run.smithi143:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-16T14:51:03.552 DEBUG:teuthology.orchestra.run.smithi143:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-16T14:51:03.634 DEBUG:teuthology.orchestra.run.smithi143:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-16T14:51:03.728 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T14:51:03.728 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-01-16T14:51:03.820 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-16T14:51:03.909 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-16T14:51:04.016 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-16T14:51:04.102 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-16T14:51:04.195 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-16T14:51:04.195 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-01-16T14:51:04.290 DEBUG:teuthology.orchestra.run.smithi143:> sudo service rsyslog restart 2024-01-16T14:51:04.299 DEBUG:teuthology.orchestra.run.smithi204:> sudo service rsyslog restart 2024-01-16T14:51:04.424 INFO:teuthology.orchestra.run.smithi143.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-16T14:51:04.459 INFO:teuthology.orchestra.run.smithi204.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-16T14:51:04.892 INFO:teuthology.run_tasks:Running task internal.timer... 2024-01-16T14:51:04.901 INFO:teuthology.task.internal:Starting timer... 2024-01-16T14:51:04.901 INFO:teuthology.run_tasks:Running task pcp... 2024-01-16T14:51:04.921 INFO:teuthology.run_tasks:Running task selinux... 2024-01-16T14:51:04.930 DEBUG:teuthology.orchestra.run.smithi143:> sudo service auditd rotate 2024-01-16T14:51:05.063 INFO:teuthology.orchestra.run.smithi143.stdout:Rotating logs: 2024-01-16T14:51:05.068 DEBUG:teuthology.orchestra.run.smithi204:> sudo service auditd rotate 2024-01-16T14:51:05.203 INFO:teuthology.orchestra.run.smithi204.stdout:Rotating logs: 2024-01-16T14:51:05.209 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-01-16T14:51:05.209 DEBUG:teuthology.orchestra.run.smithi143:> /usr/sbin/getenforce 2024-01-16T14:51:05.288 INFO:teuthology.orchestra.run.smithi143.stdout:Permissive 2024-01-16T14:51:05.288 DEBUG:teuthology.orchestra.run.smithi204:> /usr/sbin/getenforce 2024-01-16T14:51:05.347 INFO:teuthology.orchestra.run.smithi204.stdout:Permissive 2024-01-16T14:51:05.347 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi143.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi204.front.sepia.ceph.com': 'permissive'} 2024-01-16T14:51:05.348 DEBUG:teuthology.orchestra.run.smithi143:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-16T14:51:05.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-16T14:51:05.449 DEBUG:teuthology.orchestra.run.smithi204:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-16T14:51:05.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-16T14:51:05.546 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-01-16T14:51:05.546 DEBUG:teuthology.orchestra.run.smithi143:> sudo /usr/sbin/setenforce permissive 2024-01-16T14:51:05.650 DEBUG:teuthology.orchestra.run.smithi204:> sudo /usr/sbin/setenforce permissive 2024-01-16T14:51:05.743 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-01-16T14:51:05.768 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-01-16T14:51:05.785 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-01-16T14:51:05.786 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi143.front.sepia.ceph.com,smithi204.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-01-16T14:56:44.336 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi143.front.sepia.ceph.com'), Remote(name='ubuntu@smithi204.front.sepia.ceph.com')] 2024-01-16T14:56:44.338 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-01-16T14:56:44.339 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-16T14:56:44.483 DEBUG:teuthology.orchestra.run.smithi143:> true 2024-01-16T14:56:44.636 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-01-16T14:56:44.636 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-01-16T14:56:44.637 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-16T14:56:44.774 DEBUG:teuthology.orchestra.run.smithi204:> true 2024-01-16T14:56:44.924 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-01-16T14:56:44.925 INFO:teuthology.run_tasks:Running task clock... 2024-01-16T14:56:44.935 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-01-16T14:56:44.935 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-01-16T14:56:44.936 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-16T14:56:44.944 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-01-16T14:56:44.944 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-16T14:56:45.038 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-01-16T14:56:45.048 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-01-16T14:56:45.088 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-01-16T14:56:45.101 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-01-16T14:56:45.178 INFO:teuthology.orchestra.run.smithi143.stderr:sudo: ntpd: command not found 2024-01-16T14:56:45.199 INFO:teuthology.orchestra.run.smithi204.stderr:sudo: ntpd: command not found 2024-01-16T14:56:45.219 INFO:teuthology.orchestra.run.smithi143.stdout:506 Cannot talk to daemon 2024-01-16T14:56:45.243 INFO:teuthology.orchestra.run.smithi204.stdout:506 Cannot talk to daemon 2024-01-16T14:56:45.269 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-01-16T14:56:45.297 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-01-16T14:56:45.320 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-01-16T14:56:45.350 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-01-16T14:56:45.445 INFO:teuthology.orchestra.run.smithi143.stderr:bash: line 1: ntpq: command not found 2024-01-16T14:56:45.454 INFO:teuthology.orchestra.run.smithi143.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-16T14:56:45.454 INFO:teuthology.orchestra.run.smithi143.stdout:=============================================================================== 2024-01-16T14:56:45.454 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-16T14:56:45.454 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-16T14:56:45.454 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-16T14:56:45.454 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-16T14:56:45.480 INFO:teuthology.orchestra.run.smithi204.stderr:bash: line 1: ntpq: command not found 2024-01-16T14:56:45.489 INFO:teuthology.orchestra.run.smithi204.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-16T14:56:45.489 INFO:teuthology.orchestra.run.smithi204.stdout:=============================================================================== 2024-01-16T14:56:45.489 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-16T14:56:45.489 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-16T14:56:45.489 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-16T14:56:45.489 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-16T14:56:45.490 INFO:teuthology.run_tasks:Running task install... 2024-01-16T14:56:45.497 DEBUG:teuthology.task.install:project ceph 2024-01-16T14:56:45.498 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '73a4532fc4d97adee0b985d4d42874ea351a1c86'}} 2024-01-16T14:56:45.498 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '73a4532fc4d97adee0b985d4d42874ea351a1c86'} 2024-01-16T14:56:45.498 INFO:teuthology.task.install:Using flavor: default 2024-01-16T14:56:45.504 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-01-16T14:56:45.504 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-01-16T14:56:45.504 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '73a4532fc4d97adee0b985d4d42874ea351a1c86', 'tag': None, 'wait_for_package': False} 2024-01-16T14:56:45.505 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=73a4532fc4d97adee0b985d4d42874ea351a1c86 2024-01-16T14:56:45.507 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '73a4532fc4d97adee0b985d4d42874ea351a1c86', 'tag': None, 'wait_for_package': False} 2024-01-16T14:56:45.507 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=73a4532fc4d97adee0b985d4d42874ea351a1c86 2024-01-16T14:56:45.752 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/rishabh-clone-stats/73a4532fc4d97adee0b985d4d42874ea351a1c86/centos/9/flavors/default/ 2024-01-16T14:56:45.752 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7629.g73a4532f 2024-01-16T14:56:45.755 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/rishabh-clone-stats/73a4532fc4d97adee0b985d4d42874ea351a1c86/centos/9/flavors/default/ 2024-01-16T14:56:45.755 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7629.g73a4532f 2024-01-16T14:56:45.948 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-clone-stats/73a4532fc4d97adee0b985d4d42874ea351a1c86/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-clone-stats/73a4532fc4d97adee0b985d4d42874ea351a1c86/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-clone-stats/73a4532fc4d97adee0b985d4d42874ea351a1c86/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-01-16T14:56:45.948 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T14:56:45.948 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-01-16T14:56:45.974 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-clone-stats/73a4532fc4d97adee0b985d4d42874ea351a1c86/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-clone-stats/73a4532fc4d97adee0b985d4d42874ea351a1c86/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-clone-stats/73a4532fc4d97adee0b985d4d42874ea351a1c86/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-01-16T14:56:45.974 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-16T14:56:45.975 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-01-16T14:56:46.053 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-01-16T14:56:46.053 DEBUG:teuthology.orchestra.run.smithi143:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/73a4532fc4d97adee0b985d4d42874ea351a1c86/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-01-16T14:56:46.076 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-01-16T14:56:46.076 DEBUG:teuthology.orchestra.run.smithi204:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/73a4532fc4d97adee0b985d4d42874ea351a1c86/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-01-16T14:56:46.189 DEBUG:teuthology.orchestra.run.smithi143:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-01-16T14:56:46.217 DEBUG:teuthology.orchestra.run.smithi204:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-01-16T14:56:46.323 DEBUG:teuthology.orchestra.run.smithi143:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-01-16T14:56:46.352 DEBUG:teuthology.orchestra.run.smithi204:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-01-16T14:56:46.414 INFO:teuthology.orchestra.run.smithi143.stdout:check_obsoletes = 1 2024-01-16T14:56:46.419 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum clean all 2024-01-16T14:56:46.443 INFO:teuthology.orchestra.run.smithi204.stdout:check_obsoletes = 1 2024-01-16T14:56:46.448 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum clean all 2024-01-16T14:56:46.771 INFO:teuthology.orchestra.run.smithi143.stdout:73 files removed 2024-01-16T14:56:46.807 INFO:teuthology.orchestra.run.smithi204.stdout:73 files removed 2024-01-16T14:56:46.813 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-01-16T14:56:46.845 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-01-16T14:56:47.801 INFO:teuthology.orchestra.run.smithi143.stdout:ceph packages for x86_64 162 kB/s | 78 kB 00:00 2024-01-16T14:56:47.934 INFO:teuthology.orchestra.run.smithi204.stdout:ceph packages for x86_64 135 kB/s | 78 kB 00:00 2024-01-16T14:56:48.393 INFO:teuthology.orchestra.run.smithi143.stdout:ceph noarch packages 19 kB/s | 9.5 kB 00:00 2024-01-16T14:56:48.467 INFO:teuthology.orchestra.run.smithi204.stdout:ceph noarch packages 22 kB/s | 9.5 kB 00:00 2024-01-16T14:56:48.918 INFO:teuthology.orchestra.run.smithi204.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-01-16T14:56:49.094 INFO:teuthology.orchestra.run.smithi143.stdout:ceph source packages 3.0 kB/s | 1.9 kB 00:00 2024-01-16T14:56:49.838 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS-9 - AppStream 7.5 MB/s | 6.4 MB 00:00 2024-01-16T14:56:50.048 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS-9 - AppStream 7.3 MB/s | 6.4 MB 00:00 2024-01-16T14:56:51.489 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS-9 - BaseOS 4.7 MB/s | 2.0 MB 00:00 2024-01-16T14:56:51.774 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS-9 - BaseOS 3.9 MB/s | 2.0 MB 00:00 2024-01-16T14:56:54.178 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - BaseOS 4.0 MB/s | 8.0 MB 00:01 2024-01-16T14:56:56.495 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-01-16T14:56:56.526 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - BaseOS 1.7 MB/s | 8.0 MB 00:04 2024-01-16T14:57:03.071 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - CRB 2.1 MB/s | 5.9 MB 00:02 2024-01-16T14:57:04.739 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 15 kB 00:00 2024-01-16T14:57:04.827 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - AppStream 2.6 MB/s | 19 MB 00:07 2024-01-16T14:57:06.566 INFO:teuthology.orchestra.run.smithi143.stdout:Extra Packages for Enterprise Linux 12 MB/s | 20 MB 00:01 2024-01-16T14:57:12.003 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - CRB 1.7 MB/s | 5.9 MB 00:03 2024-01-16T14:57:12.207 INFO:teuthology.orchestra.run.smithi143.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-01-16T14:57:14.537 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - Extras packages 12 kB/s | 15 kB 00:01 2024-01-16T14:57:14.904 INFO:teuthology.orchestra.run.smithi143.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-01-16T14:57:14.905 INFO:teuthology.orchestra.run.smithi143.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-01-16T14:57:14.925 INFO:teuthology.orchestra.run.smithi143.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-01-16T14:57:14.926 INFO:teuthology.orchestra.run.smithi143.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-01-16T14:57:14.927 INFO:teuthology.orchestra.run.smithi143.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-01-16T14:57:14.929 INFO:teuthology.orchestra.run.smithi143.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-01-16T14:57:14.931 INFO:teuthology.orchestra.run.smithi143.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-01-16T14:57:14.935 INFO:teuthology.orchestra.run.smithi143.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-01-16T14:57:14.936 INFO:teuthology.orchestra.run.smithi143.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-01-16T14:57:15.016 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-01-16T14:57:15.031 INFO:teuthology.orchestra.run.smithi143.stdout:============================================================================================ 2024-01-16T14:57:15.031 INFO:teuthology.orchestra.run.smithi143.stdout: Package Arch Version Repository Size 2024-01-16T14:57:15.031 INFO:teuthology.orchestra.run.smithi143.stdout:============================================================================================ 2024-01-16T14:57:15.031 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2024-01-16T14:57:15.032 INFO:teuthology.orchestra.run.smithi143.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-01-16T14:57:15.032 INFO:teuthology.orchestra.run.smithi143.stdout: ceph x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 6.6 k 2024-01-16T14:57:15.032 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-base x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 5.2 M 2024-01-16T14:57:15.032 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-fuse x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 861 k 2024-01-16T14:57:15.032 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 141 k 2024-01-16T14:57:15.032 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 1.5 M 2024-01-16T14:57:15.032 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7629.g73a4532f.el9 ceph-noarch 133 k 2024-01-16T14:57:15.032 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7629.g73a4532f.el9 ceph-noarch 1.8 M 2024-01-16T14:57:15.032 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7629.g73a4532f.el9 ceph-noarch 7.4 M 2024-01-16T14:57:15.032 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-rook noarch 2:18.0.0-7629.g73a4532f.el9 ceph-noarch 50 k 2024-01-16T14:57:15.032 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-radosgw x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 8.6 M 2024-01-16T14:57:15.032 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-test x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 41 M 2024-01-16T14:57:15.032 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-volume noarch 2:18.0.0-7629.g73a4532f.el9 ceph-noarch 262 k 2024-01-16T14:57:15.033 INFO:teuthology.orchestra.run.smithi143.stdout: cephadm noarch 2:18.0.0-7629.g73a4532f.el9 ceph-noarch 778 k 2024-01-16T14:57:15.033 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-mirror x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 217 k 2024-01-16T14:57:15.033 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-top noarch 2:18.0.0-7629.g73a4532f.el9 ceph-noarch 18 k 2024-01-16T14:57:15.033 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-01-16T14:57:15.033 INFO:teuthology.orchestra.run.smithi143.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-01-16T14:57:15.033 INFO:teuthology.orchestra.run.smithi143.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-01-16T14:57:15.033 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs-devel x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 31 k 2024-01-16T14:57:15.033 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs2 x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 719 k 2024-01-16T14:57:15.033 INFO:teuthology.orchestra.run.smithi143.stdout: librados-devel x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 127 k 2024-01-16T14:57:15.033 INFO:teuthology.orchestra.run.smithi143.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-01-16T14:57:15.033 INFO:teuthology.orchestra.run.smithi143.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2024-01-16T14:57:15.033 INFO:teuthology.orchestra.run.smithi143.stdout: openssl-devel x86_64 1:3.0.7-25.el9 CentOS-AppStream 3.0 M 2024-01-16T14:57:15.033 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-01-16T14:57:15.033 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-01-16T14:57:15.034 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-01-16T14:57:15.034 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cephfs x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 161 k 2024-01-16T14:57:15.034 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rados x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 321 k 2024-01-16T14:57:15.034 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rbd x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 298 k 2024-01-16T14:57:15.034 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rgw x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 99 k 2024-01-16T14:57:15.034 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-fuse x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 85 k 2024-01-16T14:57:15.034 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-mirror x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 3.0 M 2024-01-16T14:57:15.034 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-nbd x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 171 k 2024-01-16T14:57:15.034 INFO:teuthology.orchestra.run.smithi143.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-01-16T14:57:15.034 INFO:teuthology.orchestra.run.smithi143.stdout:Upgrading: 2024-01-16T14:57:15.034 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager x86_64 1:1.45.9-1.el9 CentOS-BaseOS 2.2 M 2024-01-16T14:57:15.034 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-libnm x86_64 1:1.45.9-1.el9 CentOS-BaseOS 1.8 M 2024-01-16T14:57:15.034 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-team x86_64 1:1.45.9-1.el9 CentOS-BaseOS 42 k 2024-01-16T14:57:15.034 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-tui x86_64 1:1.45.9-1.el9 CentOS-BaseOS 246 k 2024-01-16T14:57:15.035 INFO:teuthology.orchestra.run.smithi143.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-01-16T14:57:15.035 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-01-16T14:57:15.035 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-01-16T14:57:15.035 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-01-16T14:57:15.035 INFO:teuthology.orchestra.run.smithi143.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-01-16T14:57:15.035 INFO:teuthology.orchestra.run.smithi143.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-01-16T14:57:15.035 INFO:teuthology.orchestra.run.smithi143.stdout: librados2 x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 3.3 M 2024-01-16T14:57:15.036 INFO:teuthology.orchestra.run.smithi143.stdout: librbd1 x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 3.0 M 2024-01-16T14:57:15.036 INFO:teuthology.orchestra.run.smithi143.stdout: openssl x86_64 1:3.0.7-25.el9 CentOS-BaseOS 1.2 M 2024-01-16T14:57:15.036 INFO:teuthology.orchestra.run.smithi143.stdout: openssl-libs x86_64 1:3.0.7-25.el9 CentOS-BaseOS 2.1 M 2024-01-16T14:57:15.036 INFO:teuthology.orchestra.run.smithi143.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-01-16T14:57:15.036 INFO:teuthology.orchestra.run.smithi143.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-01-16T14:57:15.036 INFO:teuthology.orchestra.run.smithi143.stdout:Installing dependencies: 2024-01-16T14:57:15.036 INFO:teuthology.orchestra.run.smithi143.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-01-16T14:57:15.036 INFO:teuthology.orchestra.run.smithi143.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-01-16T14:57:15.036 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-common x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 19 M 2024-01-16T14:57:15.036 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7629.g73a4532f.el9 ceph-noarch 28 k 2024-01-16T14:57:15.036 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mds x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 2.1 M 2024-01-16T14:57:15.036 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7629.g73a4532f.el9 ceph-noarch 246 k 2024-01-16T14:57:15.036 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mon x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 4.4 M 2024-01-16T14:57:15.037 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-osd x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 18 M 2024-01-16T14:57:15.037 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7629.g73a4532f.el9 ceph-noarch 15 k 2024-01-16T14:57:15.037 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-selinux x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 25 k 2024-01-16T14:57:15.037 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-01-16T14:57:15.037 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-01-16T14:57:15.037 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-01-16T14:57:15.037 INFO:teuthology.orchestra.run.smithi143.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-01-16T14:57:15.037 INFO:teuthology.orchestra.run.smithi143.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-01-16T14:57:15.037 INFO:teuthology.orchestra.run.smithi143.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-01-16T14:57:15.037 INFO:teuthology.orchestra.run.smithi143.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-01-16T14:57:15.037 INFO:teuthology.orchestra.run.smithi143.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-01-16T14:57:15.037 INFO:teuthology.orchestra.run.smithi143.stdout: libcephsqlite x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 163 k 2024-01-16T14:57:15.038 INFO:teuthology.orchestra.run.smithi143.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-01-16T14:57:15.038 INFO:teuthology.orchestra.run.smithi143.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-01-16T14:57:15.038 INFO:teuthology.orchestra.run.smithi143.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-01-16T14:57:15.038 INFO:teuthology.orchestra.run.smithi143.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-01-16T14:57:15.038 INFO:teuthology.orchestra.run.smithi143.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-01-16T14:57:15.038 INFO:teuthology.orchestra.run.smithi143.stdout: libradosstriper1 x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 476 k 2024-01-16T14:57:15.038 INFO:teuthology.orchestra.run.smithi143.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-01-16T14:57:15.038 INFO:teuthology.orchestra.run.smithi143.stdout: librgw2 x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 4.8 M 2024-01-16T14:57:15.038 INFO:teuthology.orchestra.run.smithi143.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-01-16T14:57:15.038 INFO:teuthology.orchestra.run.smithi143.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-01-16T14:57:15.038 INFO:teuthology.orchestra.run.smithi143.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-01-16T14:57:15.038 INFO:teuthology.orchestra.run.smithi143.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-01-16T14:57:15.039 INFO:teuthology.orchestra.run.smithi143.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-01-16T14:57:15.039 INFO:teuthology.orchestra.run.smithi143.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-01-16T14:57:15.039 INFO:teuthology.orchestra.run.smithi143.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-01-16T14:57:15.039 INFO:teuthology.orchestra.run.smithi143.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-01-16T14:57:15.039 INFO:teuthology.orchestra.run.smithi143.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-01-16T14:57:15.039 INFO:teuthology.orchestra.run.smithi143.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-01-16T14:57:15.039 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-01-16T14:57:15.039 INFO:teuthology.orchestra.run.smithi143.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-01-16T14:57:15.039 INFO:teuthology.orchestra.run.smithi143.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-01-16T14:57:15.039 INFO:teuthology.orchestra.run.smithi143.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-01-16T14:57:15.039 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-01-16T14:57:15.039 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-argparse x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 45 k 2024-01-16T14:57:15.039 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-common x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 129 k 2024-01-16T14:57:15.040 INFO:teuthology.orchestra.run.smithi143.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-01-16T14:57:15.040 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-01-16T14:57:15.040 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-01-16T14:57:15.040 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-01-16T14:57:15.040 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-01-16T14:57:15.040 INFO:teuthology.orchestra.run.smithi143.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-01-16T14:57:15.040 INFO:teuthology.orchestra.run.smithi143.stdout: python3-google-auth noarch 1:2.26.0-1.el9 epel 202 k 2024-01-16T14:57:15.040 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-01-16T14:57:15.040 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-01-16T14:57:15.040 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-01-16T14:57:15.040 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-01-16T14:57:15.040 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-01-16T14:57:15.041 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2024-01-16T14:57:15.041 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-01-16T14:57:15.041 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-01-16T14:57:15.041 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-01-16T14:57:15.041 INFO:teuthology.orchestra.run.smithi143.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-01-16T14:57:15.041 INFO:teuthology.orchestra.run.smithi143.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-01-16T14:57:15.041 INFO:teuthology.orchestra.run.smithi143.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-01-16T14:57:15.041 INFO:teuthology.orchestra.run.smithi143.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-01-16T14:57:15.041 INFO:teuthology.orchestra.run.smithi143.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-01-16T14:57:15.041 INFO:teuthology.orchestra.run.smithi143.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-01-16T14:57:15.041 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-01-16T14:57:15.041 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-01-16T14:57:15.041 INFO:teuthology.orchestra.run.smithi143.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-01-16T14:57:15.042 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-01-16T14:57:15.042 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-01-16T14:57:15.042 INFO:teuthology.orchestra.run.smithi143.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-01-16T14:57:15.042 INFO:teuthology.orchestra.run.smithi143.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-01-16T14:57:15.042 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-01-16T14:57:15.042 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-01-16T14:57:15.042 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-01-16T14:57:15.042 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-01-16T14:57:15.042 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-01-16T14:57:15.042 INFO:teuthology.orchestra.run.smithi143.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-01-16T14:57:15.042 INFO:teuthology.orchestra.run.smithi143.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2024-01-16T14:57:15.042 INFO:teuthology.orchestra.run.smithi143.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-01-16T14:57:15.042 INFO:teuthology.orchestra.run.smithi143.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-01-16T14:57:15.042 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-01-16T14:57:15.043 INFO:teuthology.orchestra.run.smithi143.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-01-16T14:57:15.043 INFO:teuthology.orchestra.run.smithi143.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-01-16T14:57:15.043 INFO:teuthology.orchestra.run.smithi143.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-01-16T14:57:15.043 INFO:teuthology.orchestra.run.smithi143.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-01-16T14:57:15.043 INFO:teuthology.orchestra.run.smithi143.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-01-16T14:57:15.043 INFO:teuthology.orchestra.run.smithi143.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-01-16T14:57:15.043 INFO:teuthology.orchestra.run.smithi143.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-01-16T14:57:15.043 INFO:teuthology.orchestra.run.smithi143.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-01-16T14:57:15.043 INFO:teuthology.orchestra.run.smithi143.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-01-16T14:57:15.043 INFO:teuthology.orchestra.run.smithi143.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-01-16T14:57:15.043 INFO:teuthology.orchestra.run.smithi143.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-01-16T14:57:15.043 INFO:teuthology.orchestra.run.smithi143.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-01-16T14:57:15.043 INFO:teuthology.orchestra.run.smithi143.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-01-16T14:57:15.043 INFO:teuthology.orchestra.run.smithi143.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-01-16T14:57:15.043 INFO:teuthology.orchestra.run.smithi143.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-01-16T14:57:15.044 INFO:teuthology.orchestra.run.smithi143.stdout:Installing weak dependencies: 2024-01-16T14:57:15.044 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-01-16T14:57:15.044 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:57:15.044 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2024-01-16T14:57:15.044 INFO:teuthology.orchestra.run.smithi143.stdout:============================================================================================ 2024-01-16T14:57:15.044 INFO:teuthology.orchestra.run.smithi143.stdout:Install 135 Packages 2024-01-16T14:57:15.044 INFO:teuthology.orchestra.run.smithi143.stdout:Upgrade 16 Packages 2024-01-16T14:57:15.044 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:57:15.045 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 204 M 2024-01-16T14:57:15.045 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2024-01-16T14:57:16.076 INFO:teuthology.orchestra.run.smithi143.stdout:(1/151): ceph-18.0.0-7629.g73a4532f.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2024-01-16T14:57:16.751 INFO:teuthology.orchestra.run.smithi143.stdout:(2/151): ceph-fuse-18.0.0-7629.g73a4532f.el9.x8 1.2 MB/s | 861 kB 00:00 2024-01-16T14:57:16.901 INFO:teuthology.orchestra.run.smithi143.stdout:(3/151): ceph-base-18.0.0-7629.g73a4532f.el9.x8 4.3 MB/s | 5.2 MB 00:01 2024-01-16T14:57:16.934 INFO:teuthology.orchestra.run.smithi143.stdout:(4/151): ceph-immutable-object-cache-18.0.0-762 772 kB/s | 141 kB 00:00 2024-01-16T14:57:17.243 INFO:teuthology.orchestra.run.smithi143.stdout:(5/151): ceph-mds-18.0.0-7629.g73a4532f.el9.x86 6.1 MB/s | 2.1 MB 00:00 2024-01-16T14:57:17.743 INFO:teuthology.orchestra.run.smithi143.stdout:(6/151): ceph-mgr-18.0.0-7629.g73a4532f.el9.x86 1.8 MB/s | 1.5 MB 00:00 2024-01-16T14:57:17.843 INFO:teuthology.orchestra.run.smithi143.stdout:(7/151): ceph-mon-18.0.0-7629.g73a4532f.el9.x86 7.3 MB/s | 4.4 MB 00:00 2024-01-16T14:57:18.802 INFO:teuthology.orchestra.run.smithi143.stdout:(8/151): ceph-radosgw-18.0.0-7629.g73a4532f.el9 9.0 MB/s | 8.6 MB 00:00 2024-01-16T14:57:18.886 INFO:teuthology.orchestra.run.smithi143.stdout:(9/151): ceph-selinux-18.0.0-7629.g73a4532f.el9 306 kB/s | 25 kB 00:00 2024-01-16T14:57:19.911 INFO:teuthology.orchestra.run.smithi143.stdout:(10/151): ceph-common-18.0.0-7629.g73a4532f.el9 4.5 MB/s | 19 MB 00:04 2024-01-16T14:57:20.061 INFO:teuthology.orchestra.run.smithi143.stdout:(11/151): cephfs-mirror-18.0.0-7629.g73a4532f.e 1.4 MB/s | 217 kB 00:00 2024-01-16T14:57:20.136 INFO:teuthology.orchestra.run.smithi143.stdout:(12/151): libcephfs-devel-18.0.0-7629.g73a4532f 420 kB/s | 31 kB 00:00 2024-01-16T14:57:20.320 INFO:teuthology.orchestra.run.smithi143.stdout:(13/151): libcephfs2-18.0.0-7629.g73a4532f.el9. 3.8 MB/s | 719 kB 00:00 2024-01-16T14:57:20.396 INFO:teuthology.orchestra.run.smithi143.stdout:(14/151): libcephsqlite-18.0.0-7629.g73a4532f.e 2.1 MB/s | 163 kB 00:00 2024-01-16T14:57:20.496 INFO:teuthology.orchestra.run.smithi143.stdout:(15/151): librados-devel-18.0.0-7629.g73a4532f. 1.2 MB/s | 127 kB 00:00 2024-01-16T14:57:20.621 INFO:teuthology.orchestra.run.smithi143.stdout:(16/151): libradosstriper1-18.0.0-7629.g73a4532 3.7 MB/s | 476 kB 00:00 2024-01-16T14:57:21.505 INFO:teuthology.orchestra.run.smithi204.stdout:Extra Packages for Enterprise Linux 3.0 MB/s | 20 MB 00:06 2024-01-16T14:57:21.830 INFO:teuthology.orchestra.run.smithi143.stdout:(17/151): librgw2-18.0.0-7629.g73a4532f.el9.x86 4.0 MB/s | 4.8 MB 00:01 2024-01-16T14:57:21.913 INFO:teuthology.orchestra.run.smithi143.stdout:(18/151): python3-ceph-argparse-18.0.0-7629.g73 549 kB/s | 45 kB 00:00 2024-01-16T14:57:22.005 INFO:teuthology.orchestra.run.smithi143.stdout:(19/151): python3-ceph-common-18.0.0-7629.g73a4 1.4 MB/s | 129 kB 00:00 2024-01-16T14:57:22.106 INFO:teuthology.orchestra.run.smithi143.stdout:(20/151): python3-cephfs-18.0.0-7629.g73a4532f. 1.6 MB/s | 161 kB 00:00 2024-01-16T14:57:22.331 INFO:teuthology.orchestra.run.smithi143.stdout:(21/151): python3-rados-18.0.0-7629.g73a4532f.e 1.4 MB/s | 321 kB 00:00 2024-01-16T14:57:22.673 INFO:teuthology.orchestra.run.smithi143.stdout:(22/151): python3-rbd-18.0.0-7629.g73a4532f.el9 875 kB/s | 298 kB 00:00 2024-01-16T14:57:22.832 INFO:teuthology.orchestra.run.smithi143.stdout:(23/151): python3-rgw-18.0.0-7629.g73a4532f.el9 628 kB/s | 99 kB 00:00 2024-01-16T14:57:22.899 INFO:teuthology.orchestra.run.smithi143.stdout:(24/151): rbd-fuse-18.0.0-7629.g73a4532f.el9.x8 1.3 MB/s | 85 kB 00:00 2024-01-16T14:57:24.516 INFO:teuthology.orchestra.run.smithi143.stdout:(25/151): rbd-mirror-18.0.0-7629.g73a4532f.el9. 1.9 MB/s | 3.0 MB 00:01 2024-01-16T14:57:24.683 INFO:teuthology.orchestra.run.smithi143.stdout:(26/151): rbd-nbd-18.0.0-7629.g73a4532f.el9.x86 1.0 MB/s | 171 kB 00:00 2024-01-16T14:57:24.750 INFO:teuthology.orchestra.run.smithi143.stdout:(27/151): ceph-grafana-dashboards-18.0.0-7629.g 426 kB/s | 28 kB 00:00 2024-01-16T14:57:24.858 INFO:teuthology.orchestra.run.smithi143.stdout:(28/151): ceph-mgr-cephadm-18.0.0-7629.g73a4532 1.2 MB/s | 133 kB 00:00 2024-01-16T14:57:25.542 INFO:teuthology.orchestra.run.smithi143.stdout:(29/151): ceph-mgr-dashboard-18.0.0-7629.g73a45 2.6 MB/s | 1.8 MB 00:00 2024-01-16T14:57:26.917 INFO:teuthology.orchestra.run.smithi143.stdout:(30/151): ceph-test-18.0.0-7629.g73a4532f.el9.x 5.1 MB/s | 41 MB 00:08 2024-01-16T14:57:27.059 INFO:teuthology.orchestra.run.smithi143.stdout:(31/151): ceph-mgr-modules-core-18.0.0-7629.g73 1.7 MB/s | 246 kB 00:00 2024-01-16T14:57:27.121 INFO:teuthology.orchestra.run.smithi204.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-01-16T14:57:27.126 INFO:teuthology.orchestra.run.smithi143.stdout:(32/151): ceph-mgr-rook-18.0.0-7629.g73a4532f.e 764 kB/s | 50 kB 00:00 2024-01-16T14:57:27.472 INFO:teuthology.orchestra.run.smithi143.stdout:(33/151): ceph-osd-18.0.0-7629.g73a4532f.el9.x8 1.8 MB/s | 18 MB 00:09 2024-01-16T14:57:27.502 INFO:teuthology.orchestra.run.smithi143.stdout:(34/151): ceph-prometheus-alerts-18.0.0-7629.g7 40 kB/s | 15 kB 00:00 2024-01-16T14:57:27.652 INFO:teuthology.orchestra.run.smithi143.stdout:(35/151): ceph-volume-18.0.0-7629.g73a4532f.el9 1.4 MB/s | 262 kB 00:00 2024-01-16T14:57:27.727 INFO:teuthology.orchestra.run.smithi143.stdout:(36/151): cephfs-top-18.0.0-7629.g73a4532f.el9. 246 kB/s | 18 kB 00:00 2024-01-16T14:57:27.780 INFO:teuthology.orchestra.run.smithi143.stdout:(37/151): cephadm-18.0.0-7629.g73a4532f.el9.noa 2.7 MB/s | 778 kB 00:00 2024-01-16T14:57:28.034 INFO:teuthology.orchestra.run.smithi143.stdout:(38/151): automake-1.16.2-8.el9.noarch.rpm 2.1 MB/s | 662 kB 00:00 2024-01-16T14:57:28.097 INFO:teuthology.orchestra.run.smithi143.stdout:(39/151): bison-3.7.4-5.el9.x86_64.rpm 3.6 MB/s | 920 kB 00:00 2024-01-16T14:57:28.122 INFO:teuthology.orchestra.run.smithi143.stdout:(40/151): boost-program-options-1.75.0-8.el9.x8 1.2 MB/s | 106 kB 00:00 2024-01-16T14:57:28.156 INFO:teuthology.orchestra.run.smithi143.stdout:(41/151): elfutils-libelf-devel-0.190-2.el9.x86 355 kB/s | 20 kB 00:00 2024-01-16T14:57:28.189 INFO:teuthology.orchestra.run.smithi143.stdout:(42/151): flex-2.6.4-9.el9.x86_64.rpm 4.5 MB/s | 307 kB 00:00 2024-01-16T14:57:28.215 INFO:teuthology.orchestra.run.smithi143.stdout:(43/151): flexiblas-3.0.4-8.el9.x86_64.rpm 572 kB/s | 32 kB 00:00 2024-01-16T14:57:28.323 INFO:teuthology.orchestra.run.smithi143.stdout:(44/151): flexiblas-netlib-3.0.4-8.el9.x86_64.r 23 MB/s | 3.0 MB 00:00 2024-01-16T14:57:28.348 INFO:teuthology.orchestra.run.smithi143.stdout:(45/151): flexiblas-openblas-openmp-3.0.4-8.el9 136 kB/s | 17 kB 00:00 2024-01-16T14:57:28.482 INFO:teuthology.orchestra.run.smithi143.stdout:(46/151): ceph-mgr-diskprediction-local-18.0.0- 2.5 MB/s | 7.4 MB 00:02 2024-01-16T14:57:28.507 INFO:teuthology.orchestra.run.smithi143.stdout:(47/151): libaio-devel-0.3.111-13.el9.x86_64.rp 64 kB/s | 10 kB 00:00 2024-01-16T14:57:28.532 INFO:teuthology.orchestra.run.smithi143.stdout:(48/151): libpmemobj-1.12.1-1.el9.x86_64.rpm 872 kB/s | 159 kB 00:00 2024-01-16T14:57:28.558 INFO:teuthology.orchestra.run.smithi143.stdout:(49/151): librabbitmq-0.11.0-7.el9.x86_64.rpm 604 kB/s | 44 kB 00:00 2024-01-16T14:57:28.629 INFO:teuthology.orchestra.run.smithi143.stdout:(50/151): libtool-2.4.6-45.el9.x86_64.rpm 6.0 MB/s | 578 kB 00:00 2024-01-16T14:57:28.675 INFO:teuthology.orchestra.run.smithi143.stdout:(51/151): librdkafka-1.6.1-102.el9.x86_64.rpm 4.6 MB/s | 661 kB 00:00 2024-01-16T14:57:28.700 INFO:teuthology.orchestra.run.smithi143.stdout:(52/151): libuuid-devel-2.37.4-15.el9.x86_64.rp 179 kB/s | 24 kB 00:00 2024-01-16T14:57:28.725 INFO:teuthology.orchestra.run.smithi143.stdout:(53/151): libzstd-devel-1.5.1-2.el9.x86_64.rpm 515 kB/s | 46 kB 00:00 2024-01-16T14:57:28.759 INFO:teuthology.orchestra.run.smithi143.stdout:(54/151): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 282 kB 00:00 2024-01-16T14:57:28.784 INFO:teuthology.orchestra.run.smithi143.stdout:(55/151): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 187 kB 00:00 2024-01-16T14:57:28.809 INFO:teuthology.orchestra.run.smithi143.stdout:(56/151): openblas-0.3.21-2.el9.x86_64.rpm 431 kB/s | 34 kB 00:00 2024-01-16T14:57:28.926 INFO:teuthology.orchestra.run.smithi143.stdout:(57/151): openssl-devel-3.0.7-25.el9.x86_64.rpm 21 MB/s | 3.0 MB 00:00 2024-01-16T14:57:29.085 INFO:teuthology.orchestra.run.smithi143.stdout:(58/151): openblas-openmp-0.3.21-2.el9.x86_64.r 16 MB/s | 4.7 MB 00:00 2024-01-16T14:57:29.183 INFO:teuthology.orchestra.run.smithi143.stdout:(59/151): postgresql-13.11-1.el9.x86_64.rpm 4.2 MB/s | 1.5 MB 00:00 2024-01-16T14:57:29.227 INFO:teuthology.orchestra.run.smithi143.stdout:(60/151): postgresql-contrib-13.11-1.el9.x86_64 2.8 MB/s | 811 kB 00:00 2024-01-16T14:57:29.260 INFO:teuthology.orchestra.run.smithi143.stdout:(61/151): postgresql-private-libs-13.11-1.el9.x 825 kB/s | 134 kB 00:00 2024-01-16T14:57:29.603 INFO:teuthology.orchestra.run.smithi143.stdout:(62/151): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 5.8 MB 00:00 2024-01-16T14:57:29.719 INFO:teuthology.orchestra.run.smithi143.stdout:(63/151): postgresql-server-13.11-1.el9.x86_64. 12 MB/s | 5.7 MB 00:00 2024-01-16T14:57:29.769 INFO:teuthology.orchestra.run.smithi143.stdout:(64/151): python3-jinja2-2.11.3-4.el9.noarch.rp 462 kB/s | 228 kB 00:00 2024-01-16T14:57:29.786 INFO:teuthology.orchestra.run.smithi143.stdout:(65/151): python3-jsonpatch-1.21-16.el9.noarch. 143 kB/s | 24 kB 00:00 2024-01-16T14:57:29.828 INFO:teuthology.orchestra.run.smithi143.stdout:(66/151): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 144 kB 00:00 2024-01-16T14:57:29.845 INFO:teuthology.orchestra.run.smithi143.stdout:(67/151): python3-jsonpointer-2.0-4.el9.noarch. 244 kB/s | 18 kB 00:00 2024-01-16T14:57:29.862 INFO:teuthology.orchestra.run.smithi143.stdout:(68/151): python3-markupsafe-1.1.1-12.el9.x86_6 421 kB/s | 32 kB 00:00 2024-01-16T14:57:29.870 INFO:teuthology.orchestra.run.smithi204.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-01-16T14:57:29.872 INFO:teuthology.orchestra.run.smithi204.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-01-16T14:57:29.892 INFO:teuthology.orchestra.run.smithi204.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-01-16T14:57:29.893 INFO:teuthology.orchestra.run.smithi204.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-01-16T14:57:29.894 INFO:teuthology.orchestra.run.smithi204.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-01-16T14:57:29.896 INFO:teuthology.orchestra.run.smithi204.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-01-16T14:57:29.898 INFO:teuthology.orchestra.run.smithi204.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-01-16T14:57:29.902 INFO:teuthology.orchestra.run.smithi204.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-01-16T14:57:29.903 INFO:teuthology.orchestra.run.smithi204.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-01-16T14:57:29.987 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-01-16T14:57:30.007 INFO:teuthology.orchestra.run.smithi204.stdout:============================================================================================ 2024-01-16T14:57:30.007 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-01-16T14:57:30.007 INFO:teuthology.orchestra.run.smithi204.stdout:============================================================================================ 2024-01-16T14:57:30.007 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-01-16T14:57:30.007 INFO:teuthology.orchestra.run.smithi204.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-01-16T14:57:30.007 INFO:teuthology.orchestra.run.smithi204.stdout: ceph x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 6.6 k 2024-01-16T14:57:30.007 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 5.2 M 2024-01-16T14:57:30.007 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 861 k 2024-01-16T14:57:30.007 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 141 k 2024-01-16T14:57:30.007 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 1.5 M 2024-01-16T14:57:30.007 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7629.g73a4532f.el9 ceph-noarch 133 k 2024-01-16T14:57:30.007 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7629.g73a4532f.el9 ceph-noarch 1.8 M 2024-01-16T14:57:30.007 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7629.g73a4532f.el9 ceph-noarch 7.4 M 2024-01-16T14:57:30.008 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook noarch 2:18.0.0-7629.g73a4532f.el9 ceph-noarch 50 k 2024-01-16T14:57:30.008 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 8.6 M 2024-01-16T14:57:30.008 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 41 M 2024-01-16T14:57:30.008 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume noarch 2:18.0.0-7629.g73a4532f.el9 ceph-noarch 262 k 2024-01-16T14:57:30.008 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm noarch 2:18.0.0-7629.g73a4532f.el9 ceph-noarch 778 k 2024-01-16T14:57:30.008 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-mirror x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 217 k 2024-01-16T14:57:30.008 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-top noarch 2:18.0.0-7629.g73a4532f.el9 ceph-noarch 18 k 2024-01-16T14:57:30.008 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-01-16T14:57:30.008 INFO:teuthology.orchestra.run.smithi204.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-01-16T14:57:30.008 INFO:teuthology.orchestra.run.smithi204.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-01-16T14:57:30.008 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 31 k 2024-01-16T14:57:30.008 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2 x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 719 k 2024-01-16T14:57:30.008 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 127 k 2024-01-16T14:57:30.008 INFO:teuthology.orchestra.run.smithi204.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-01-16T14:57:30.009 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2024-01-16T14:57:30.009 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-devel x86_64 1:3.0.7-25.el9 CentOS-AppStream 3.0 M 2024-01-16T14:57:30.009 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-01-16T14:57:30.009 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-01-16T14:57:30.009 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-01-16T14:57:30.009 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 161 k 2024-01-16T14:57:30.009 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 321 k 2024-01-16T14:57:30.009 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 298 k 2024-01-16T14:57:30.009 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 99 k 2024-01-16T14:57:30.009 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 85 k 2024-01-16T14:57:30.009 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 3.0 M 2024-01-16T14:57:30.009 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 171 k 2024-01-16T14:57:30.009 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-01-16T14:57:30.009 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrading: 2024-01-16T14:57:30.010 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager x86_64 1:1.45.9-1.el9 CentOS-BaseOS 2.2 M 2024-01-16T14:57:30.010 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-libnm x86_64 1:1.45.9-1.el9 CentOS-BaseOS 1.8 M 2024-01-16T14:57:30.010 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-team x86_64 1:1.45.9-1.el9 CentOS-BaseOS 42 k 2024-01-16T14:57:30.010 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-tui x86_64 1:1.45.9-1.el9 CentOS-BaseOS 246 k 2024-01-16T14:57:30.010 INFO:teuthology.orchestra.run.smithi204.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-01-16T14:57:30.010 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-01-16T14:57:30.010 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-01-16T14:57:30.010 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-01-16T14:57:30.010 INFO:teuthology.orchestra.run.smithi204.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-01-16T14:57:30.010 INFO:teuthology.orchestra.run.smithi204.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-01-16T14:57:30.010 INFO:teuthology.orchestra.run.smithi204.stdout: librados2 x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 3.3 M 2024-01-16T14:57:30.010 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1 x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 3.0 M 2024-01-16T14:57:30.010 INFO:teuthology.orchestra.run.smithi204.stdout: openssl x86_64 1:3.0.7-25.el9 CentOS-BaseOS 1.2 M 2024-01-16T14:57:30.010 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-libs x86_64 1:3.0.7-25.el9 CentOS-BaseOS 2.1 M 2024-01-16T14:57:30.010 INFO:teuthology.orchestra.run.smithi204.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-01-16T14:57:30.011 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-01-16T14:57:30.011 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-01-16T14:57:30.011 INFO:teuthology.orchestra.run.smithi204.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-01-16T14:57:30.011 INFO:teuthology.orchestra.run.smithi204.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-01-16T14:57:30.011 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 19 M 2024-01-16T14:57:30.011 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7629.g73a4532f.el9 ceph-noarch 28 k 2024-01-16T14:57:30.011 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 2.1 M 2024-01-16T14:57:30.011 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7629.g73a4532f.el9 ceph-noarch 246 k 2024-01-16T14:57:30.011 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 4.4 M 2024-01-16T14:57:30.011 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 18 M 2024-01-16T14:57:30.011 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7629.g73a4532f.el9 ceph-noarch 15 k 2024-01-16T14:57:30.011 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 25 k 2024-01-16T14:57:30.011 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-01-16T14:57:30.011 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-01-16T14:57:30.011 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-01-16T14:57:30.011 INFO:teuthology.orchestra.run.smithi204.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-01-16T14:57:30.011 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-01-16T14:57:30.012 INFO:teuthology.orchestra.run.smithi204.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-01-16T14:57:30.012 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-01-16T14:57:30.012 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-01-16T14:57:30.012 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 163 k 2024-01-16T14:57:30.012 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-01-16T14:57:30.012 INFO:teuthology.orchestra.run.smithi204.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-01-16T14:57:30.012 INFO:teuthology.orchestra.run.smithi204.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-01-16T14:57:30.012 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-01-16T14:57:30.012 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-01-16T14:57:30.012 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1 x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 476 k 2024-01-16T14:57:30.012 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-01-16T14:57:30.012 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2 x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 4.8 M 2024-01-16T14:57:30.013 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-01-16T14:57:30.013 INFO:teuthology.orchestra.run.smithi204.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-01-16T14:57:30.013 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-01-16T14:57:30.013 INFO:teuthology.orchestra.run.smithi204.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-01-16T14:57:30.013 INFO:teuthology.orchestra.run.smithi204.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-01-16T14:57:30.013 INFO:teuthology.orchestra.run.smithi204.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-01-16T14:57:30.013 INFO:teuthology.orchestra.run.smithi204.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-01-16T14:57:30.013 INFO:teuthology.orchestra.run.smithi204.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-01-16T14:57:30.013 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-01-16T14:57:30.013 INFO:teuthology.orchestra.run.smithi204.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-01-16T14:57:30.013 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-01-16T14:57:30.013 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-01-16T14:57:30.013 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-01-16T14:57:30.013 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-01-16T14:57:30.013 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-01-16T14:57:30.013 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 45 k 2024-01-16T14:57:30.014 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common x86_64 2:18.0.0-7629.g73a4532f.el9 ceph 129 k 2024-01-16T14:57:30.014 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-01-16T14:57:30.014 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-01-16T14:57:30.014 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-01-16T14:57:30.014 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-01-16T14:57:30.014 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-01-16T14:57:30.014 INFO:teuthology.orchestra.run.smithi204.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-01-16T14:57:30.014 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth noarch 1:2.26.0-1.el9 epel 202 k 2024-01-16T14:57:30.014 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-01-16T14:57:30.014 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-01-16T14:57:30.014 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-01-16T14:57:30.014 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-01-16T14:57:30.014 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-01-16T14:57:30.014 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2024-01-16T14:57:30.014 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-01-16T14:57:30.015 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-01-16T14:57:30.015 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-01-16T14:57:30.015 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-01-16T14:57:30.015 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-01-16T14:57:30.015 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-01-16T14:57:30.015 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-01-16T14:57:30.015 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-01-16T14:57:30.015 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-01-16T14:57:30.015 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-01-16T14:57:30.015 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-01-16T14:57:30.015 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-01-16T14:57:30.015 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-01-16T14:57:30.015 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-01-16T14:57:30.015 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-01-16T14:57:30.015 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-01-16T14:57:30.015 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-01-16T14:57:30.015 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-01-16T14:57:30.016 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-01-16T14:57:30.016 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-01-16T14:57:30.016 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-01-16T14:57:30.016 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-01-16T14:57:30.016 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2024-01-16T14:57:30.016 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-01-16T14:57:30.016 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-01-16T14:57:30.016 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-01-16T14:57:30.016 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-01-16T14:57:30.016 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-01-16T14:57:30.016 INFO:teuthology.orchestra.run.smithi204.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-01-16T14:57:30.016 INFO:teuthology.orchestra.run.smithi204.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-01-16T14:57:30.016 INFO:teuthology.orchestra.run.smithi204.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-01-16T14:57:30.016 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-01-16T14:57:30.017 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-01-16T14:57:30.017 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-01-16T14:57:30.017 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-01-16T14:57:30.017 INFO:teuthology.orchestra.run.smithi204.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-01-16T14:57:30.017 INFO:teuthology.orchestra.run.smithi204.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-01-16T14:57:30.017 INFO:teuthology.orchestra.run.smithi204.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-01-16T14:57:30.017 INFO:teuthology.orchestra.run.smithi204.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-01-16T14:57:30.017 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-01-16T14:57:30.017 INFO:teuthology.orchestra.run.smithi204.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-01-16T14:57:30.017 INFO:teuthology.orchestra.run.smithi204.stdout:Installing weak dependencies: 2024-01-16T14:57:30.017 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-01-16T14:57:30.017 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:57:30.017 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-01-16T14:57:30.017 INFO:teuthology.orchestra.run.smithi204.stdout:============================================================================================ 2024-01-16T14:57:30.017 INFO:teuthology.orchestra.run.smithi204.stdout:Install 135 Packages 2024-01-16T14:57:30.017 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrade 16 Packages 2024-01-16T14:57:30.018 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:57:30.018 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 204 M 2024-01-16T14:57:30.018 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-01-16T14:57:30.096 INFO:teuthology.orchestra.run.smithi143.stdout:(69/151): python3-numpy-1.20.1-5.el9.x86_64.rpm 18 MB/s | 4.8 MB 00:00 2024-01-16T14:57:30.121 INFO:teuthology.orchestra.run.smithi143.stdout:(70/151): python3-numpy-f2py-1.20.1-5.el9.x86_6 854 kB/s | 217 kB 00:00 2024-01-16T14:57:30.146 INFO:teuthology.orchestra.run.smithi143.stdout:(71/151): python3-oauthlib-3.1.1-5.el9.noarch.r 624 kB/s | 174 kB 00:00 2024-01-16T14:57:30.188 INFO:teuthology.orchestra.run.smithi143.stdout:(72/151): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 41 kB 00:00 2024-01-16T14:57:30.222 INFO:teuthology.orchestra.run.smithi143.stdout:(73/151): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.8 MB/s | 132 kB 00:00 2024-01-16T14:57:30.247 INFO:teuthology.orchestra.run.smithi143.stdout:(74/151): python3-pyasn1-modules-0.4.8-6.el9.no 2.1 MB/s | 211 kB 00:00 2024-01-16T14:57:30.272 INFO:teuthology.orchestra.run.smithi143.stdout:(75/151): python3-pytz-2021.1-5.el9.noarch.rpm 585 kB/s | 47 kB 00:00 2024-01-16T14:57:30.306 INFO:teuthology.orchestra.run.smithi143.stdout:(76/151): python3-requests-oauthlib-1.3.0-12.el 749 kB/s | 43 kB 00:00 2024-01-16T14:57:30.589 INFO:teuthology.orchestra.run.smithi143.stdout:(77/151): python3-toml-0.10.2-6.el9.noarch.rpm 126 kB/s | 39 kB 00:00 2024-01-16T14:57:30.790 INFO:teuthology.orchestra.run.smithi143.stdout:(78/151): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-01-16T14:57:30.814 INFO:teuthology.orchestra.run.smithi143.stdout:(79/151): socat-1.7.4.1-5.el9.x86_64.rpm 608 kB/s | 300 kB 00:00 2024-01-16T14:57:30.840 INFO:teuthology.orchestra.run.smithi143.stdout:(80/151): uuid-1.6.2-55.el9.x86_64.rpm 231 kB/s | 56 kB 00:00 2024-01-16T14:57:30.865 INFO:teuthology.orchestra.run.smithi143.stdout:(81/151): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2024-01-16T14:57:30.915 INFO:teuthology.orchestra.run.smithi143.stdout:(82/151): xmlstarlet-1.6.1-20.el9.x86_64.rpm 634 kB/s | 63 kB 00:00 2024-01-16T14:57:30.941 INFO:teuthology.orchestra.run.smithi143.stdout:(83/151): zlib-devel-1.2.11-40.el9.x86_64.rpm 448 kB/s | 44 kB 00:00 2024-01-16T14:57:30.948 INFO:teuthology.orchestra.run.smithi204.stdout:(1/151): ceph-18.0.0-7629.g73a4532f.el9.x86_64. 26 kB/s | 6.6 kB 00:00 2024-01-16T14:57:30.966 INFO:teuthology.orchestra.run.smithi143.stdout:(84/151): fuse-2.9.9-15.el9.x86_64.rpm 1.6 MB/s | 79 kB 00:00 2024-01-16T14:57:30.991 INFO:teuthology.orchestra.run.smithi143.stdout:(85/151): jq-1.6-16.el9.x86_64.rpm 2.5 MB/s | 186 kB 00:00 2024-01-16T14:57:31.041 INFO:teuthology.orchestra.run.smithi143.stdout:(86/151): libgfortran-11.4.1-3.el9.x86_64.rpm 8.0 MB/s | 802 kB 00:00 2024-01-16T14:57:31.075 INFO:teuthology.orchestra.run.smithi143.stdout:(87/151): libquadmath-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 194 kB 00:00 2024-01-16T14:57:31.100 INFO:teuthology.orchestra.run.smithi143.stdout:(88/151): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.1 MB/s | 217 kB 00:00 2024-01-16T14:57:31.134 INFO:teuthology.orchestra.run.smithi143.stdout:(89/151): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 241 kB 00:00 2024-01-16T14:57:31.173 INFO:teuthology.orchestra.run.smithi204.stdout:(2/151): ceph-fuse-18.0.0-7629.g73a4532f.el9.x8 3.8 MB/s | 861 kB 00:00 2024-01-16T14:57:31.176 INFO:teuthology.orchestra.run.smithi143.stdout:(90/151): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 103 kB 00:00 2024-01-16T14:57:31.226 INFO:teuthology.orchestra.run.smithi143.stdout:(91/151): python3-cryptography-36.0.1-4.el9.x86 9.3 MB/s | 1.1 MB 00:00 2024-01-16T14:57:31.251 INFO:teuthology.orchestra.run.smithi143.stdout:(92/151): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 124 kB 00:00 2024-01-16T14:57:31.265 INFO:teuthology.orchestra.run.smithi204.stdout:(3/151): ceph-immutable-object-cache-18.0.0-762 1.5 MB/s | 141 kB 00:00 2024-01-16T14:57:31.276 INFO:teuthology.orchestra.run.smithi143.stdout:(93/151): python3-requests-2.25.1-7.el9.noarch. 1.2 MB/s | 113 kB 00:00 2024-01-16T14:57:31.318 INFO:teuthology.orchestra.run.smithi143.stdout:(94/151): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 187 kB 00:00 2024-01-16T14:57:31.507 INFO:teuthology.orchestra.run.smithi143.stdout:(95/151): lua-devel-5.4.4-4.el9.x86_64.rpm 98 kB/s | 22 kB 00:00 2024-01-16T14:57:31.544 INFO:teuthology.orchestra.run.smithi143.stdout:(96/151): python3-devel-3.9.17-2.el9.x86_64.rpm 846 kB/s | 245 kB 00:00 2024-01-16T14:57:31.549 INFO:teuthology.orchestra.run.smithi204.stdout:(4/151): ceph-base-18.0.0-7629.g73a4532f.el9.x8 6.1 MB/s | 5.2 MB 00:00 2024-01-16T14:57:31.641 INFO:teuthology.orchestra.run.smithi204.stdout:(5/151): ceph-mds-18.0.0-7629.g73a4532f.el9.x86 5.6 MB/s | 2.1 MB 00:00 2024-01-16T14:57:31.733 INFO:teuthology.orchestra.run.smithi204.stdout:(6/151): ceph-mgr-18.0.0-7629.g73a4532f.el9.x86 8.0 MB/s | 1.5 MB 00:00 2024-01-16T14:57:31.853 INFO:teuthology.orchestra.run.smithi143.stdout:(97/151): gperftools-libs-2.9.1-2.el9.x86_64.rp 579 kB/s | 309 kB 00:00 2024-01-16T14:57:31.877 INFO:teuthology.orchestra.run.smithi143.stdout:(98/151): libarrow-doc-9.0.0-10.el9.noarch.rpm 78 kB/s | 26 kB 00:00 2024-01-16T14:57:31.936 INFO:teuthology.orchestra.run.smithi143.stdout:(99/151): liboath-2.6.7-2.el9.x86_64.rpm 588 kB/s | 49 kB 00:00 2024-01-16T14:57:32.028 INFO:teuthology.orchestra.run.smithi143.stdout:(100/151): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-01-16T14:57:32.087 INFO:teuthology.orchestra.run.smithi143.stdout:(101/151): libunwind-1.6.2-1.el9.x86_64.rpm 323 kB/s | 67 kB 00:00 2024-01-16T14:57:32.212 INFO:teuthology.orchestra.run.smithi143.stdout:(102/151): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.5 MB/s | 838 kB 00:00 2024-01-16T14:57:32.275 INFO:teuthology.orchestra.run.smithi204.stdout:(7/151): ceph-mon-18.0.0-7629.g73a4532f.el9.x86 6.9 MB/s | 4.4 MB 00:00 2024-01-16T14:57:32.321 INFO:teuthology.orchestra.run.smithi143.stdout:(103/151): libarrow-9.0.0-10.el9.x86_64.rpm 5.5 MB/s | 4.4 MB 00:00 2024-01-16T14:57:32.346 INFO:teuthology.orchestra.run.smithi143.stdout:(104/151): python3-bcrypt-3.2.2-1.el9.x86_64.rp 327 kB/s | 43 kB 00:00 2024-01-16T14:57:32.380 INFO:teuthology.orchestra.run.smithi143.stdout:(105/151): python3-asyncssh-2.13.2-3.el9.noarch 1.8 MB/s | 548 kB 00:00 2024-01-16T14:57:32.405 INFO:teuthology.orchestra.run.smithi143.stdout:(106/151): python3-cachetools-4.2.4-1.el9.noarc 388 kB/s | 32 kB 00:00 2024-01-16T14:57:32.430 INFO:teuthology.orchestra.run.smithi143.stdout:(107/151): python3-certifi-2021.10.8-2.el9.noar 180 kB/s | 15 kB 00:00 2024-01-16T14:57:32.572 INFO:teuthology.orchestra.run.smithi143.stdout:(108/151): python3-cheroot-8.6.0-4.el9.noarch.r 897 kB/s | 172 kB 00:00 2024-01-16T14:57:32.606 INFO:teuthology.orchestra.run.smithi143.stdout:(109/151): python3-google-auth-2.26.0-1.el9.noa 1.1 MB/s | 202 kB 00:00 2024-01-16T14:57:32.665 INFO:teuthology.orchestra.run.smithi143.stdout:(110/151): python3-cherrypy-18.6.1-2.el9.noarch 1.4 MB/s | 358 kB 00:00 2024-01-16T14:57:32.690 INFO:teuthology.orchestra.run.smithi143.stdout:(111/151): python3-jaraco-8.2.1-3.el9.noarch.rp 92 kB/s | 11 kB 00:00 2024-01-16T14:57:32.715 INFO:teuthology.orchestra.run.smithi143.stdout:(112/151): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-01-16T14:57:32.740 INFO:teuthology.orchestra.run.smithi143.stdout:(113/151): python3-jaraco-collections-3.0.0-8.e 310 kB/s | 23 kB 00:00 2024-01-16T14:57:32.765 INFO:teuthology.orchestra.run.smithi143.stdout:(114/151): python3-jaraco-functools-3.5.0-2.el9 260 kB/s | 19 kB 00:00 2024-01-16T14:57:32.791 INFO:teuthology.orchestra.run.smithi143.stdout:(115/151): python3-jaraco-text-3.2.0-6.el9.noar 263 kB/s | 20 kB 00:00 2024-01-16T14:57:32.816 INFO:teuthology.orchestra.run.smithi143.stdout:(116/151): python3-jwt+crypto-2.4.0-1.el9.noarc 121 kB/s | 9.0 kB 00:00 2024-01-16T14:57:32.849 INFO:teuthology.orchestra.run.smithi143.stdout:(117/151): python3-jwt-2.4.0-1.el9.noarch.rpm 492 kB/s | 41 kB 00:00 2024-01-16T14:57:32.891 INFO:teuthology.orchestra.run.smithi143.stdout:(118/151): python3-logutils-0.3.5-21.el9.noarch 621 kB/s | 46 kB 00:00 2024-01-16T14:57:32.933 INFO:teuthology.orchestra.run.smithi143.stdout:(119/151): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-01-16T14:57:32.959 INFO:teuthology.orchestra.run.smithi143.stdout:(120/151): python3-more-itertools-8.12.0-2.el9. 729 kB/s | 79 kB 00:00 2024-01-16T14:57:32.984 INFO:teuthology.orchestra.run.smithi143.stdout:(121/151): python3-natsort-7.1.1-5.el9.noarch.r 631 kB/s | 58 kB 00:00 2024-01-16T14:57:33.025 INFO:teuthology.orchestra.run.smithi204.stdout:(8/151): ceph-common-18.0.0-7629.g73a4532f.el9. 8.2 MB/s | 19 MB 00:02 2024-01-16T14:57:33.026 INFO:teuthology.orchestra.run.smithi143.stdout:(122/151): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-01-16T14:57:33.051 INFO:teuthology.orchestra.run.smithi143.stdout:(123/151): python3-portend-3.1.0-2.el9.noarch.r 179 kB/s | 16 kB 00:00 2024-01-16T14:57:33.076 INFO:teuthology.orchestra.run.smithi143.stdout:(124/151): python3-pyOpenSSL-21.0.0-1.el9.noarc 984 kB/s | 90 kB 00:00 2024-01-16T14:57:33.100 INFO:teuthology.orchestra.run.smithi204.stdout:(9/151): ceph-selinux-18.0.0-7629.g73a4532f.el9 341 kB/s | 25 kB 00:00 2024-01-16T14:57:33.102 INFO:teuthology.orchestra.run.smithi143.stdout:(125/151): python3-repoze-lru-0.7-16.el9.noarch 409 kB/s | 31 kB 00:00 2024-01-16T14:57:33.135 INFO:teuthology.orchestra.run.smithi143.stdout:(126/151): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-01-16T14:57:33.160 INFO:teuthology.orchestra.run.smithi143.stdout:(127/151): python3-rsa-4.9-2.el9.noarch.rpm 712 kB/s | 59 kB 00:00 2024-01-16T14:57:33.185 INFO:teuthology.orchestra.run.smithi143.stdout:(128/151): python3-tempora-5.0.0-2.el9.noarch.r 431 kB/s | 36 kB 00:00 2024-01-16T14:57:33.236 INFO:teuthology.orchestra.run.smithi143.stdout:(129/151): python3-typing-extensions-4.4.0-2.el 511 kB/s | 51 kB 00:00 2024-01-16T14:57:33.261 INFO:teuthology.orchestra.run.smithi143.stdout:(130/151): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-01-16T14:57:33.295 INFO:teuthology.orchestra.run.smithi143.stdout:(131/151): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-01-16T14:57:33.328 INFO:teuthology.orchestra.run.smithi143.stdout:(132/151): python3-werkzeug-2.0.3-3.el9.1.noarc 4.6 MB/s | 427 kB 00:00 2024-01-16T14:57:33.362 INFO:teuthology.orchestra.run.smithi143.stdout:(133/151): python3-zc-lockfile-2.0-10.el9.noarc 199 kB/s | 20 kB 00:00 2024-01-16T14:57:33.396 INFO:teuthology.orchestra.run.smithi143.stdout:(134/151): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-01-16T14:57:33.409 INFO:teuthology.orchestra.run.smithi204.stdout:(10/151): ceph-radosgw-18.0.0-7629.g73a4532f.el 7.6 MB/s | 8.6 MB 00:01 2024-01-16T14:57:33.471 INFO:teuthology.orchestra.run.smithi143.stdout:(135/151): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-01-16T14:57:33.513 INFO:teuthology.orchestra.run.smithi143.stdout:(136/151): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.6 MB/s | 68 kB 00:00 2024-01-16T14:57:33.643 INFO:teuthology.orchestra.run.smithi204.stdout:(11/151): ceph-osd-18.0.0-7629.g73a4532f.el9.x8 9.3 MB/s | 18 MB 00:01 2024-01-16T14:57:33.655 INFO:teuthology.orchestra.run.smithi143.stdout:(137/151): NetworkManager-1.45.9-1.el9.x86_64.r 16 MB/s | 2.2 MB 00:00 2024-01-16T14:57:33.676 INFO:teuthology.orchestra.run.smithi204.stdout:(12/151): cephfs-mirror-18.0.0-7629.g73a4532f.e 816 kB/s | 217 kB 00:00 2024-01-16T14:57:33.718 INFO:teuthology.orchestra.run.smithi204.stdout:(13/151): libcephfs-devel-18.0.0-7629.g73a4532f 420 kB/s | 31 kB 00:00 2024-01-16T14:57:33.809 INFO:teuthology.orchestra.run.smithi204.stdout:(14/151): libcephfs2-18.0.0-7629.g73a4532f.el9. 5.4 MB/s | 719 kB 00:00 2024-01-16T14:57:33.844 INFO:teuthology.orchestra.run.smithi204.stdout:(15/151): libcephsqlite-18.0.0-7629.g73a4532f.e 1.3 MB/s | 163 kB 00:00 2024-01-16T14:57:33.877 INFO:teuthology.orchestra.run.smithi204.stdout:(16/151): librados-devel-18.0.0-7629.g73a4532f. 1.8 MB/s | 127 kB 00:00 2024-01-16T14:57:33.888 INFO:teuthology.orchestra.run.smithi143.stdout:(138/151): NetworkManager-libnm-1.45.9-1.el9.x8 7.6 MB/s | 1.8 MB 00:00 2024-01-16T14:57:33.914 INFO:teuthology.orchestra.run.smithi143.stdout:(139/151): NetworkManager-team-1.45.9-1.el9.x86 1.7 MB/s | 42 kB 00:00 2024-01-16T14:57:33.944 INFO:teuthology.orchestra.run.smithi204.stdout:(17/151): libradosstriper1-18.0.0-7629.g73a4532 4.7 MB/s | 476 kB 00:00 2024-01-16T14:57:33.972 INFO:teuthology.orchestra.run.smithi143.stdout:(140/151): NetworkManager-tui-1.45.9-1.el9.x86_ 4.2 MB/s | 246 kB 00:00 2024-01-16T14:57:34.019 INFO:teuthology.orchestra.run.smithi204.stdout:(18/151): python3-ceph-argparse-18.0.0-7629.g73 609 kB/s | 45 kB 00:00 2024-01-16T14:57:34.023 INFO:teuthology.orchestra.run.smithi143.stdout:(141/151): acl-2.3.1-4.el9.x86_64.rpm 1.4 MB/s | 69 kB 00:00 2024-01-16T14:57:34.056 INFO:teuthology.orchestra.run.smithi143.stdout:(142/151): elfutils-debuginfod-client-0.190-2.e 1.1 MB/s | 35 kB 00:00 2024-01-16T14:57:34.111 INFO:teuthology.orchestra.run.smithi204.stdout:(19/151): python3-ceph-common-18.0.0-7629.g73a4 1.4 MB/s | 129 kB 00:00 2024-01-16T14:57:34.115 INFO:teuthology.orchestra.run.smithi143.stdout:(143/151): elfutils-libelf-0.190-2.el9.x86_64.r 3.2 MB/s | 191 kB 00:00 2024-01-16T14:57:34.173 INFO:teuthology.orchestra.run.smithi143.stdout:(144/151): elfutils-libs-0.190-2.el9.x86_64.rpm 4.3 MB/s | 254 kB 00:00 2024-01-16T14:57:34.195 INFO:teuthology.orchestra.run.smithi204.stdout:(20/151): python3-cephfs-18.0.0-7629.g73a4532f. 1.9 MB/s | 161 kB 00:00 2024-01-16T14:57:34.215 INFO:teuthology.orchestra.run.smithi143.stdout:(145/151): libacl-2.3.1-4.el9.x86_64.rpm 534 kB/s | 22 kB 00:00 2024-01-16T14:57:34.287 INFO:teuthology.orchestra.run.smithi204.stdout:(21/151): python3-rados-18.0.0-7629.g73a4532f.e 3.5 MB/s | 321 kB 00:00 2024-01-16T14:57:34.299 INFO:teuthology.orchestra.run.smithi143.stdout:(146/151): openssl-3.0.7-25.el9.x86_64.rpm 14 MB/s | 1.2 MB 00:00 2024-01-16T14:57:34.421 INFO:teuthology.orchestra.run.smithi204.stdout:(22/151): librgw2-18.0.0-7629.g73a4532f.el9.x86 8.8 MB/s | 4.8 MB 00:00 2024-01-16T14:57:34.432 INFO:teuthology.orchestra.run.smithi143.stdout:(147/151): openssl-libs-3.0.7-25.el9.x86_64.rpm 16 MB/s | 2.1 MB 00:00 2024-01-16T14:57:34.454 INFO:teuthology.orchestra.run.smithi204.stdout:(23/151): python3-rbd-18.0.0-7629.g73a4532f.el9 1.8 MB/s | 298 kB 00:00 2024-01-16T14:57:34.488 INFO:teuthology.orchestra.run.smithi204.stdout:(24/151): python3-rgw-18.0.0-7629.g73a4532f.el9 1.5 MB/s | 99 kB 00:00 2024-01-16T14:57:34.491 INFO:teuthology.orchestra.run.smithi143.stdout:(148/151): xfsdump-3.1.12-4.el9.x86_64.rpm 5.4 MB/s | 321 kB 00:00 2024-01-16T14:57:34.513 INFO:teuthology.orchestra.run.smithi204.stdout:(25/151): rbd-fuse-18.0.0-7629.g73a4532f.el9.x8 1.4 MB/s | 85 kB 00:00 2024-01-16T14:57:34.583 INFO:teuthology.orchestra.run.smithi143.stdout:(149/151): xfsprogs-6.3.0-1.el9.x86_64.rpm 12 MB/s | 1.1 MB 00:00 2024-01-16T14:57:34.605 INFO:teuthology.orchestra.run.smithi204.stdout:(26/151): rbd-nbd-18.0.0-7629.g73a4532f.el9.x86 1.8 MB/s | 171 kB 00:00 2024-01-16T14:57:34.680 INFO:teuthology.orchestra.run.smithi204.stdout:(27/151): ceph-grafana-dashboards-18.0.0-7629.g 378 kB/s | 28 kB 00:00 2024-01-16T14:57:34.764 INFO:teuthology.orchestra.run.smithi204.stdout:(28/151): ceph-mgr-cephadm-18.0.0-7629.g73a4532 1.6 MB/s | 133 kB 00:00 2024-01-16T14:57:34.856 INFO:teuthology.orchestra.run.smithi204.stdout:(29/151): rbd-mirror-18.0.0-7629.g73a4532f.el9. 8.2 MB/s | 3.0 MB 00:00 2024-01-16T14:57:34.990 INFO:teuthology.orchestra.run.smithi204.stdout:(30/151): ceph-mgr-dashboard-18.0.0-7629.g73a45 7.8 MB/s | 1.8 MB 00:00 2024-01-16T14:57:35.074 INFO:teuthology.orchestra.run.smithi204.stdout:(31/151): ceph-mgr-modules-core-18.0.0-7629.g73 2.9 MB/s | 246 kB 00:00 2024-01-16T14:57:35.092 INFO:teuthology.orchestra.run.smithi143.stdout:(150/151): librbd1-18.0.0-7629.g73a4532f.el9.x8 1.8 MB/s | 3.0 MB 00:01 2024-01-16T14:57:35.149 INFO:teuthology.orchestra.run.smithi204.stdout:(32/151): ceph-mgr-rook-18.0.0-7629.g73a4532f.e 678 kB/s | 50 kB 00:00 2024-01-16T14:57:35.224 INFO:teuthology.orchestra.run.smithi204.stdout:(33/151): ceph-prometheus-alerts-18.0.0-7629.g7 204 kB/s | 15 kB 00:00 2024-01-16T14:57:35.316 INFO:teuthology.orchestra.run.smithi204.stdout:(34/151): ceph-volume-18.0.0-7629.g73a4532f.el9 2.8 MB/s | 262 kB 00:00 2024-01-16T14:57:35.475 INFO:teuthology.orchestra.run.smithi204.stdout:(35/151): cephadm-18.0.0-7629.g73a4532f.el9.noa 4.8 MB/s | 778 kB 00:00 2024-01-16T14:57:35.566 INFO:teuthology.orchestra.run.smithi143.stdout:(151/151): librados2-18.0.0-7629.g73a4532f.el9. 1.5 MB/s | 3.3 MB 00:02 2024-01-16T14:57:35.581 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2024-01-16T14:57:35.581 INFO:teuthology.orchestra.run.smithi143.stdout:Total 9.9 MB/s | 204 MB 00:20 2024-01-16T14:57:35.617 INFO:teuthology.orchestra.run.smithi204.stdout:(36/151): ceph-mgr-diskprediction-local-18.0.0- 9.7 MB/s | 7.4 MB 00:00 2024-01-16T14:57:35.642 INFO:teuthology.orchestra.run.smithi204.stdout:(37/151): cephfs-top-18.0.0-7629.g73a4532f.el9. 110 kB/s | 18 kB 00:00 2024-01-16T14:57:35.669 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2024-01-16T14:57:36.016 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2024-01-16T14:57:36.017 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2024-01-16T14:57:36.051 INFO:teuthology.orchestra.run.smithi204.stdout:(38/151): automake-1.16.2-8.el9.noarch.rpm 1.5 MB/s | 662 kB 00:00 2024-01-16T14:57:36.126 INFO:teuthology.orchestra.run.smithi204.stdout:(39/151): bison-3.7.4-5.el9.x86_64.rpm 1.9 MB/s | 920 kB 00:00 2024-01-16T14:57:36.151 INFO:teuthology.orchestra.run.smithi204.stdout:(40/151): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 106 kB 00:00 2024-01-16T14:57:36.176 INFO:teuthology.orchestra.run.smithi204.stdout:(41/151): elfutils-libelf-devel-0.190-2.el9.x86 420 kB/s | 20 kB 00:00 2024-01-16T14:57:36.219 INFO:teuthology.orchestra.run.smithi204.stdout:(42/151): flex-2.6.4-9.el9.x86_64.rpm 4.5 MB/s | 307 kB 00:00 2024-01-16T14:57:36.244 INFO:teuthology.orchestra.run.smithi204.stdout:(43/151): flexiblas-3.0.4-8.el9.x86_64.rpm 477 kB/s | 32 kB 00:00 2024-01-16T14:57:36.369 INFO:teuthology.orchestra.run.smithi204.stdout:(44/151): flexiblas-openblas-openmp-3.0.4-8.el9 138 kB/s | 17 kB 00:00 2024-01-16T14:57:36.495 INFO:teuthology.orchestra.run.smithi204.stdout:(45/151): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-01-16T14:57:36.520 INFO:teuthology.orchestra.run.smithi204.stdout:(46/151): libaio-devel-0.3.111-13.el9.x86_64.rp 70 kB/s | 10 kB 00:00 2024-01-16T14:57:36.554 INFO:teuthology.orchestra.run.smithi204.stdout:(47/151): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 159 kB 00:00 2024-01-16T14:57:36.578 INFO:teuthology.orchestra.run.smithi204.stdout:(48/151): librabbitmq-0.11.0-7.el9.x86_64.rpm 762 kB/s | 44 kB 00:00 2024-01-16T14:57:36.637 INFO:teuthology.orchestra.run.smithi204.stdout:(49/151): librdkafka-1.6.1-102.el9.x86_64.rpm 7.8 MB/s | 661 kB 00:00 2024-01-16T14:57:36.704 INFO:teuthology.orchestra.run.smithi204.stdout:(50/151): libtool-2.4.6-45.el9.x86_64.rpm 4.5 MB/s | 578 kB 00:00 2024-01-16T14:57:36.730 INFO:teuthology.orchestra.run.smithi204.stdout:(51/151): libuuid-devel-2.37.4-15.el9.x86_64.rp 284 kB/s | 24 kB 00:00 2024-01-16T14:57:36.755 INFO:teuthology.orchestra.run.smithi204.stdout:(52/151): libzstd-devel-1.5.1-2.el9.x86_64.rpm 937 kB/s | 46 kB 00:00 2024-01-16T14:57:36.789 INFO:teuthology.orchestra.run.smithi204.stdout:(53/151): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 282 kB 00:00 2024-01-16T14:57:36.822 INFO:teuthology.orchestra.run.smithi204.stdout:(54/151): lua-5.4.4-4.el9.x86_64.rpm 2.9 MB/s | 187 kB 00:00 2024-01-16T14:57:36.839 INFO:teuthology.orchestra.run.smithi204.stdout:(55/151): openblas-0.3.21-2.el9.x86_64.rpm 712 kB/s | 34 kB 00:00 2024-01-16T14:57:37.356 INFO:teuthology.orchestra.run.smithi204.stdout:(56/151): ceph-test-18.0.0-7629.g73a4532f.el9.x 9.7 MB/s | 41 MB 00:04 2024-01-16T14:57:37.542 INFO:teuthology.orchestra.run.smithi204.stdout:(57/151): openblas-openmp-0.3.21-2.el9.x86_64.r 6.6 MB/s | 4.7 MB 00:00 2024-01-16T14:57:37.642 INFO:teuthology.orchestra.run.smithi204.stdout:(58/151): openssl-devel-3.0.7-25.el9.x86_64.rpm 3.7 MB/s | 3.0 MB 00:00 2024-01-16T14:57:37.701 INFO:teuthology.orchestra.run.smithi204.stdout:(59/151): postgresql-13.11-1.el9.x86_64.rpm 4.8 MB/s | 1.5 MB 00:00 2024-01-16T14:57:37.734 INFO:teuthology.orchestra.run.smithi204.stdout:(60/151): postgresql-contrib-13.11-1.el9.x86_64 4.4 MB/s | 811 kB 00:00 2024-01-16T14:57:37.760 INFO:teuthology.orchestra.run.smithi204.stdout:(61/151): postgresql-private-libs-13.11-1.el9.x 1.2 MB/s | 134 kB 00:00 2024-01-16T14:57:37.985 INFO:teuthology.orchestra.run.smithi204.stdout:(62/151): python3-babel-2.9.1-2.el9.noarch.rpm 23 MB/s | 5.8 MB 00:00 2024-01-16T14:57:38.044 INFO:teuthology.orchestra.run.smithi204.stdout:(63/151): python3-jinja2-2.11.3-4.el9.noarch.rp 824 kB/s | 228 kB 00:00 2024-01-16T14:57:38.161 INFO:teuthology.orchestra.run.smithi204.stdout:(64/151): postgresql-server-13.11-1.el9.x86_64. 13 MB/s | 5.7 MB 00:00 2024-01-16T14:57:38.186 INFO:teuthology.orchestra.run.smithi204.stdout:(65/151): python3-jsonpatch-1.21-16.el9.noarch. 130 kB/s | 24 kB 00:00 2024-01-16T14:57:38.197 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2024-01-16T14:57:38.198 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2024-01-16T14:57:38.211 INFO:teuthology.orchestra.run.smithi204.stdout:(66/151): python3-jsonpointer-2.0-4.el9.noarch. 116 kB/s | 18 kB 00:00 2024-01-16T14:57:38.236 INFO:teuthology.orchestra.run.smithi204.stdout:(67/151): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 144 kB 00:00 2024-01-16T14:57:38.261 INFO:teuthology.orchestra.run.smithi204.stdout:(68/151): python3-markupsafe-1.1.1-12.el9.x86_6 427 kB/s | 32 kB 00:00 2024-01-16T14:57:38.421 INFO:teuthology.orchestra.run.smithi204.stdout:(69/151): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 4.8 MB 00:00 2024-01-16T14:57:38.479 INFO:teuthology.orchestra.run.smithi204.stdout:(70/151): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.0 MB/s | 217 kB 00:00 2024-01-16T14:57:38.504 INFO:teuthology.orchestra.run.smithi204.stdout:(71/151): python3-oauthlib-3.1.1-5.el9.noarch.r 767 kB/s | 174 kB 00:00 2024-01-16T14:57:38.529 INFO:teuthology.orchestra.run.smithi204.stdout:(72/151): python3-prettytable-0.7.2-27.el9.noar 390 kB/s | 41 kB 00:00 2024-01-16T14:57:38.563 INFO:teuthology.orchestra.run.smithi204.stdout:(73/151): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 132 kB 00:00 2024-01-16T14:57:38.588 INFO:teuthology.orchestra.run.smithi204.stdout:(74/151): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 211 kB 00:00 2024-01-16T14:57:38.613 INFO:teuthology.orchestra.run.smithi204.stdout:(75/151): python3-pytz-2021.1-5.el9.noarch.rpm 590 kB/s | 47 kB 00:00 2024-01-16T14:57:38.647 INFO:teuthology.orchestra.run.smithi204.stdout:(76/151): python3-requests-oauthlib-1.3.0-12.el 749 kB/s | 43 kB 00:00 2024-01-16T14:57:38.864 INFO:teuthology.orchestra.run.smithi204.stdout:(77/151): python3-toml-0.10.2-6.el9.noarch.rpm 161 kB/s | 39 kB 00:00 2024-01-16T14:57:38.897 INFO:teuthology.orchestra.run.smithi204.stdout:(78/151): socat-1.7.4.1-5.el9.x86_64.rpm 1.2 MB/s | 300 kB 00:00 2024-01-16T14:57:39.140 INFO:teuthology.orchestra.run.smithi204.stdout:(79/151): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-01-16T14:57:39.165 INFO:teuthology.orchestra.run.smithi204.stdout:(80/151): uuid-1.6.2-55.el9.x86_64.rpm 189 kB/s | 56 kB 00:00 2024-01-16T14:57:39.190 INFO:teuthology.orchestra.run.smithi204.stdout:(81/151): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 312 kB/s | 87 kB 00:00 2024-01-16T14:57:39.215 INFO:teuthology.orchestra.run.smithi204.stdout:(82/151): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 63 kB 00:00 2024-01-16T14:57:39.240 INFO:teuthology.orchestra.run.smithi204.stdout:(83/151): zlib-devel-1.2.11-40.el9.x86_64.rpm 602 kB/s | 44 kB 00:00 2024-01-16T14:57:39.266 INFO:teuthology.orchestra.run.smithi204.stdout:(84/151): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-01-16T14:57:39.300 INFO:teuthology.orchestra.run.smithi204.stdout:(85/151): jq-1.6-16.el9.x86_64.rpm 3.2 MB/s | 186 kB 00:00 2024-01-16T14:57:39.341 INFO:teuthology.orchestra.run.smithi204.stdout:(86/151): libgfortran-11.4.1-3.el9.x86_64.rpm 7.9 MB/s | 802 kB 00:00 2024-01-16T14:57:39.375 INFO:teuthology.orchestra.run.smithi204.stdout:(87/151): libquadmath-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 194 kB 00:00 2024-01-16T14:57:39.409 INFO:teuthology.orchestra.run.smithi204.stdout:(88/151): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.3 MB/s | 217 kB 00:00 2024-01-16T14:57:39.434 INFO:teuthology.orchestra.run.smithi204.stdout:(89/151): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.6 MB/s | 241 kB 00:00 2024-01-16T14:57:39.576 INFO:teuthology.orchestra.run.smithi204.stdout:(90/151): python3-cryptography-36.0.1-4.el9.x86 5.8 MB/s | 1.1 MB 00:00 2024-01-16T14:57:39.601 INFO:teuthology.orchestra.run.smithi204.stdout:(91/151): python3-ply-3.11-14.el9.noarch.rpm 658 kB/s | 103 kB 00:00 2024-01-16T14:57:39.626 INFO:teuthology.orchestra.run.smithi204.stdout:(92/151): python3-pycparser-2.20-6.el9.noarch.r 684 kB/s | 124 kB 00:00 2024-01-16T14:57:39.693 INFO:teuthology.orchestra.run.smithi204.stdout:(93/151): python3-requests-2.25.1-7.el9.noarch. 1.7 MB/s | 113 kB 00:00 2024-01-16T14:57:39.727 INFO:teuthology.orchestra.run.smithi204.stdout:(94/151): python3-urllib3-1.26.5-5.el9.noarch.r 1.8 MB/s | 187 kB 00:00 2024-01-16T14:57:40.119 INFO:teuthology.orchestra.run.smithi204.stdout:(95/151): gperftools-libs-2.9.1-2.el9.x86_64.rp 790 kB/s | 309 kB 00:00 2024-01-16T14:57:40.153 INFO:teuthology.orchestra.run.smithi204.stdout:(96/151): lua-devel-5.4.4-4.el9.x86_64.rpm 49 kB/s | 22 kB 00:00 2024-01-16T14:57:40.336 INFO:teuthology.orchestra.run.smithi204.stdout:(97/151): libarrow-doc-9.0.0-10.el9.noarch.rpm 142 kB/s | 26 kB 00:00 2024-01-16T14:57:40.487 INFO:teuthology.orchestra.run.smithi204.stdout:(98/151): libarrow-9.0.0-10.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-01-16T14:57:40.512 INFO:teuthology.orchestra.run.smithi204.stdout:(99/151): liboath-2.6.7-2.el9.x86_64.rpm 278 kB/s | 49 kB 00:00 2024-01-16T14:57:40.587 INFO:teuthology.orchestra.run.smithi204.stdout:(100/151): libunwind-1.6.2-1.el9.x86_64.rpm 678 kB/s | 67 kB 00:00 2024-01-16T14:57:40.621 INFO:teuthology.orchestra.run.smithi204.stdout:(101/151): python3-devel-3.9.17-2.el9.x86_64.rp 247 kB/s | 245 kB 00:00 2024-01-16T14:57:40.705 INFO:teuthology.orchestra.run.smithi204.stdout:(102/151): luarocks-3.9.2-1.el9.noarch.rpm 788 kB/s | 151 kB 00:00 2024-01-16T14:57:40.746 INFO:teuthology.orchestra.run.smithi204.stdout:(103/151): parquet-libs-9.0.0-10.el9.x86_64.rpm 5.2 MB/s | 838 kB 00:00 2024-01-16T14:57:40.863 INFO:teuthology.orchestra.run.smithi204.stdout:(104/151): python3-cachetools-4.2.4-1.el9.noarc 277 kB/s | 32 kB 00:00 2024-01-16T14:57:40.888 INFO:teuthology.orchestra.run.smithi204.stdout:(105/151): python3-bcrypt-3.2.2-1.el9.x86_64.rp 237 kB/s | 43 kB 00:00 2024-01-16T14:57:40.947 INFO:teuthology.orchestra.run.smithi204.stdout:(106/151): python3-certifi-2021.10.8-2.el9.noar 181 kB/s | 15 kB 00:00 2024-01-16T14:57:41.147 INFO:teuthology.orchestra.run.smithi204.stdout:(107/151): python3-cherrypy-18.6.1-2.el9.noarch 1.8 MB/s | 358 kB 00:00 2024-01-16T14:57:41.198 INFO:teuthology.orchestra.run.smithi204.stdout:(108/151): python3-asyncssh-2.13.2-3.el9.noarch 952 kB/s | 548 kB 00:00 2024-01-16T14:57:41.223 INFO:teuthology.orchestra.run.smithi204.stdout:(109/151): python3-cheroot-8.6.0-4.el9.noarch.r 515 kB/s | 172 kB 00:00 2024-01-16T14:57:41.431 INFO:teuthology.orchestra.run.smithi204.stdout:(110/151): python3-google-auth-2.26.0-1.el9.noa 713 kB/s | 202 kB 00:00 2024-01-16T14:57:41.457 INFO:teuthology.orchestra.run.smithi204.stdout:(111/151): python3-jaraco-8.2.1-3.el9.noarch.rp 41 kB/s | 11 kB 00:00 2024-01-16T14:57:41.482 INFO:teuthology.orchestra.run.smithi204.stdout:(112/151): python3-jaraco-classes-3.2.1-5.el9.n 68 kB/s | 18 kB 00:00 2024-01-16T14:57:41.549 INFO:teuthology.orchestra.run.smithi204.stdout:(113/151): python3-jaraco-collections-3.0.0-8.e 199 kB/s | 23 kB 00:00 2024-01-16T14:57:41.641 INFO:teuthology.orchestra.run.smithi204.stdout:(114/151): python3-jaraco-functools-3.5.0-2.el9 106 kB/s | 19 kB 00:00 2024-01-16T14:57:41.666 INFO:teuthology.orchestra.run.smithi204.stdout:(115/151): python3-jwt+crypto-2.4.0-1.el9.noarc 78 kB/s | 9.0 kB 00:00 2024-01-16T14:57:41.692 INFO:teuthology.orchestra.run.smithi204.stdout:(116/151): python3-jaraco-text-3.2.0-6.el9.noar 94 kB/s | 20 kB 00:00 2024-01-16T14:57:41.775 INFO:teuthology.orchestra.run.smithi204.stdout:(117/151): python3-jwt-2.4.0-1.el9.noarch.rpm 307 kB/s | 41 kB 00:00 2024-01-16T14:57:41.842 INFO:teuthology.orchestra.run.smithi204.stdout:(118/151): python3-logutils-0.3.5-21.el9.noarch 309 kB/s | 46 kB 00:00 2024-01-16T14:57:41.876 INFO:teuthology.orchestra.run.smithi204.stdout:(119/151): python3-more-itertools-8.12.0-2.el9. 791 kB/s | 79 kB 00:00 2024-01-16T14:57:41.951 INFO:teuthology.orchestra.run.smithi204.stdout:(120/151): python3-natsort-7.1.1-5.el9.noarch.r 534 kB/s | 58 kB 00:00 2024-01-16T14:57:42.043 INFO:teuthology.orchestra.run.smithi204.stdout:(121/151): python3-portend-3.1.0-2.el9.noarch.r 180 kB/s | 16 kB 00:00 2024-01-16T14:57:42.277 INFO:teuthology.orchestra.run.smithi204.stdout:(122/151): python3-pecan-1.4.2-3.el9.noarch.rpm 681 kB/s | 272 kB 00:00 2024-01-16T14:57:42.369 INFO:teuthology.orchestra.run.smithi204.stdout:(123/151): python3-kubernetes-26.1.0-2.el9.noar 1.5 MB/s | 1.0 MB 00:00 2024-01-16T14:57:42.386 INFO:teuthology.orchestra.run.smithi204.stdout:(124/151): python3-repoze-lru-0.7-16.el9.noarch 284 kB/s | 31 kB 00:00 2024-01-16T14:57:42.419 INFO:teuthology.orchestra.run.smithi204.stdout:(125/151): python3-pyOpenSSL-21.0.0-1.el9.noarc 240 kB/s | 90 kB 00:00 2024-01-16T14:57:42.478 INFO:teuthology.orchestra.run.smithi204.stdout:(126/151): python3-rsa-4.9-2.el9.noarch.rpm 647 kB/s | 59 kB 00:00 2024-01-16T14:57:42.709 INFO:teuthology.orchestra.run.smithi204.stdout:(127/151): python3-typing-extensions-4.4.0-2.el 221 kB/s | 51 kB 00:00 2024-01-16T14:57:42.737 INFO:teuthology.orchestra.run.smithi204.stdout:(128/151): python3-tempora-5.0.0-2.el9.noarch.r 113 kB/s | 36 kB 00:00 2024-01-16T14:57:42.771 INFO:teuthology.orchestra.run.smithi204.stdout:(129/151): python3-routes-2.5.1-5.el9.noarch.rp 470 kB/s | 188 kB 00:00 2024-01-16T14:57:42.846 INFO:teuthology.orchestra.run.smithi204.stdout:(130/151): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-01-16T14:57:42.871 INFO:teuthology.orchestra.run.smithi204.stdout:(131/151): python3-websocket-client-1.2.3-2.el9 673 kB/s | 90 kB 00:00 2024-01-16T14:57:42.938 INFO:teuthology.orchestra.run.smithi204.stdout:(132/151): python3-zc-lockfile-2.0-10.el9.noarc 219 kB/s | 20 kB 00:00 2024-01-16T14:57:42.988 INFO:teuthology.orchestra.run.smithi204.stdout:(133/151): python3-werkzeug-2.0.3-3.el9.1.noarc 1.9 MB/s | 427 kB 00:00 2024-01-16T14:57:43.072 INFO:teuthology.orchestra.run.smithi204.stdout:(134/151): re2-20211101-3.el9.x86_64.rpm 958 kB/s | 192 kB 00:00 2024-01-16T14:57:43.151 INFO:teuthology.orchestra.run.smithi204.stdout:(135/151): thrift-0.15.0-2.el9.x86_64.rpm 7.5 MB/s | 1.6 MB 00:00 2024-01-16T14:57:43.214 INFO:teuthology.orchestra.run.smithi204.stdout:(136/151): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.1 MB/s | 68 kB 00:00 2024-01-16T14:57:43.356 INFO:teuthology.orchestra.run.smithi204.stdout:(137/151): librados2-18.0.0-7629.g73a4532f.el9. 9.0 MB/s | 3.3 MB 00:00 2024-01-16T14:57:43.474 INFO:teuthology.orchestra.run.smithi204.stdout:(138/151): librbd1-18.0.0-7629.g73a4532f.el9.x8 7.6 MB/s | 3.0 MB 00:00 2024-01-16T14:57:43.824 INFO:teuthology.orchestra.run.smithi204.stdout:(139/151): NetworkManager-1.45.9-1.el9.x86_64.r 3.7 MB/s | 2.2 MB 00:00 2024-01-16T14:57:44.133 INFO:teuthology.orchestra.run.smithi204.stdout:(140/151): NetworkManager-libnm-1.45.9-1.el9.x8 2.3 MB/s | 1.8 MB 00:00 2024-01-16T14:57:44.157 INFO:teuthology.orchestra.run.smithi204.stdout:(141/151): NetworkManager-team-1.45.9-1.el9.x86 62 kB/s | 42 kB 00:00 2024-01-16T14:57:44.183 INFO:teuthology.orchestra.run.smithi204.stdout:(142/151): NetworkManager-tui-1.45.9-1.el9.x86_ 692 kB/s | 246 kB 00:00 2024-01-16T14:57:44.208 INFO:teuthology.orchestra.run.smithi204.stdout:(143/151): acl-2.3.1-4.el9.x86_64.rpm 956 kB/s | 69 kB 00:00 2024-01-16T14:57:44.242 INFO:teuthology.orchestra.run.smithi204.stdout:(144/151): elfutils-debuginfod-client-0.190-2.e 618 kB/s | 35 kB 00:00 2024-01-16T14:57:44.308 INFO:teuthology.orchestra.run.smithi204.stdout:(145/151): elfutils-libelf-0.190-2.el9.x86_64.r 1.5 MB/s | 191 kB 00:00 2024-01-16T14:57:44.342 INFO:teuthology.orchestra.run.smithi204.stdout:(146/151): elfutils-libs-0.190-2.el9.x86_64.rpm 1.9 MB/s | 254 kB 00:00 2024-01-16T14:57:44.359 INFO:teuthology.orchestra.run.smithi204.stdout:(147/151): libacl-2.3.1-4.el9.x86_64.rpm 193 kB/s | 22 kB 00:00 2024-01-16T14:57:44.510 INFO:teuthology.orchestra.run.smithi204.stdout:(148/151): openssl-3.0.7-25.el9.x86_64.rpm 5.8 MB/s | 1.2 MB 00:00 2024-01-16T14:57:44.524 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2024-01-16T14:57:44.610 INFO:teuthology.orchestra.run.smithi204.stdout:(149/151): xfsdump-3.1.12-4.el9.x86_64.rpm 1.3 MB/s | 321 kB 00:00 2024-01-16T14:57:44.701 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/167 2024-01-16T14:57:44.760 INFO:teuthology.orchestra.run.smithi204.stdout:(150/151): openssl-libs-3.0.7-25.el9.x86_64.rpm 5.4 MB/s | 2.1 MB 00:00 2024-01-16T14:57:44.827 INFO:teuthology.orchestra.run.smithi204.stdout:(151/151): xfsprogs-6.3.0-1.el9.x86_64.rpm 3.4 MB/s | 1.1 MB 00:00 2024-01-16T14:57:44.839 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-01-16T14:57:44.839 INFO:teuthology.orchestra.run.smithi204.stdout:Total 14 MB/s | 204 MB 00:14 2024-01-16T14:57:44.916 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : thrift-0.15.0-2.el9.x86_64 2/167 2024-01-16T14:57:44.931 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-01-16T14:57:45.042 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/167 2024-01-16T14:57:45.125 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 4/167 2024-01-16T14:57:45.160 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/167 2024-01-16T14:57:45.241 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/167 2024-01-16T14:57:45.277 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-01-16T14:57:45.277 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-01-16T14:57:45.632 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/167 2024-01-16T14:57:45.681 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : librados2-2:18.0.0-7629.g73a4532f.el9.x86_64 8/167 2024-01-16T14:57:45.828 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librados2-2:18.0.0-7629.g73a4532f.el9.x86_64 8/167 2024-01-16T14:57:45.857 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libcephfs2-2:18.0.0-7629.g73a4532f.el9.x86_64 9/167 2024-01-16T14:57:45.985 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libcephfs2-2:18.0.0-7629.g73a4532f.el9.x86_64 9/167 2024-01-16T14:57:46.025 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rados-2:18.0.0-7629.g73a4532f.el9.x86_64 10/167 2024-01-16T14:57:46.177 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/167 2024-01-16T14:57:46.286 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 12/167 2024-01-16T14:57:46.570 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 13/167 2024-01-16T14:57:46.629 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : postgresql-13.11-1.el9.x86_64 14/167 2024-01-16T14:57:46.655 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libcephsqlite-2:18.0.0-7629.g73a4532f.el9.x86_64 15/167 2024-01-16T14:57:46.802 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7629.g73a4532f.el9.x86_64 15/167 2024-01-16T14:57:46.831 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libradosstriper1-2:18.0.0-7629.g73a4532f.el9.x86 16/167 2024-01-16T14:57:46.897 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7629.g73a4532f.el9.x86 16/167 2024-01-16T14:57:47.220 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 17/167 2024-01-16T14:57:47.402 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : NetworkManager-libnm-1:1.45.9-1.el9.x86_64 18/167 2024-01-16T14:57:47.417 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-01-16T14:57:47.418 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-01-16T14:57:47.490 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.9-1.el9.x86_64 18/167 2024-01-16T14:57:47.675 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: NetworkManager-1:1.45.9-1.el9.x86_64 19/167 2024-01-16T14:57:47.713 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : NetworkManager-1:1.45.9-1.el9.x86_64 19/167 2024-01-16T14:57:48.124 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: NetworkManager-1:1.45.9-1.el9.x86_64 19/167 2024-01-16T14:57:48.160 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 20/167 2024-01-16T14:57:48.297 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/167 2024-01-16T14:57:48.369 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 22/167 2024-01-16T14:57:48.406 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 23/167 2024-01-16T14:57:48.467 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 24/167 2024-01-16T14:57:48.517 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 25/167 2024-01-16T14:57:48.621 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 26/167 2024-01-16T14:57:48.675 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 27/167 2024-01-16T14:57:48.720 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/167 2024-01-16T14:57:48.785 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : lua-5.4.4-4.el9.x86_64 29/167 2024-01-16T14:57:48.913 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 30/167 2024-01-16T14:57:48.956 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-ceph-common-2:18.0.0-7629.g73a4532f.el9. 31/167 2024-01-16T14:57:49.011 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-ceph-argparse-2:18.0.0-7629.g73a4532f.el 32/167 2024-01-16T14:57:49.046 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cephfs-2:18.0.0-7629.g73a4532f.el9.x86_6 33/167 2024-01-16T14:57:49.165 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/167 2024-01-16T14:57:49.265 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/167 2024-01-16T14:57:49.438 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/167 2024-01-16T14:57:49.499 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/167 2024-01-16T14:57:50.190 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/167 2024-01-16T14:57:50.279 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/167 2024-01-16T14:57:50.323 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 40/167 2024-01-16T14:57:50.372 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 41/167 2024-01-16T14:57:50.411 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librados-devel-2:18.0.0-7629.g73a4532f.el9.x86_6 42/167 2024-01-16T14:57:50.469 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/167 2024-01-16T14:57:50.682 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 44/167 2024-01-16T14:57:50.723 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 45/167 2024-01-16T14:57:50.859 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : re2-1:20211101-3.el9.x86_64 46/167 2024-01-16T14:57:50.893 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 47/167 2024-01-16T14:57:50.943 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/167 2024-01-16T14:57:51.020 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 49/167 2024-01-16T14:57:51.083 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 50/167 2024-01-16T14:57:51.125 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 51/167 2024-01-16T14:57:51.191 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 52/167 2024-01-16T14:57:51.250 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 53/167 2024-01-16T14:57:51.310 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/167 2024-01-16T14:57:51.500 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/167 2024-01-16T14:57:51.551 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/167 2024-01-16T14:57:51.594 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 57/167 2024-01-16T14:57:51.749 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 58/167 2024-01-16T14:57:51.791 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-google-auth-1:2.26.0-1.el9.noarch 59/167 2024-01-16T14:57:51.839 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 60/167 2024-01-16T14:57:51.871 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 61/167 2024-01-16T14:57:52.020 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 62/167 2024-01-16T14:57:52.068 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 63/167 2024-01-16T14:57:52.136 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-ply-3.11-14.el9.noarch 64/167 2024-01-16T14:57:52.216 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 65/167 2024-01-16T14:57:52.475 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 66/167 2024-01-16T14:57:52.529 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 67/167 2024-01-16T14:57:52.703 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 68/167 2024-01-16T14:57:52.757 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 69/167 2024-01-16T14:57:52.814 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 70/167 2024-01-16T14:57:52.858 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 71/167 2024-01-16T14:57:52.909 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 72/167 2024-01-16T14:57:52.941 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 73/167 2024-01-16T14:57:52.988 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : jq-1.6-16.el9.x86_64 74/167 2024-01-16T14:57:53.029 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : fuse-2.9.9-15.el9.x86_64 75/167 2024-01-16T14:57:53.080 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 76/167 2024-01-16T14:57:53.145 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 77/167 2024-01-16T14:57:53.213 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : uuid-1.6.2-55.el9.x86_64 78/167 2024-01-16T14:57:53.262 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 79/167 2024-01-16T14:57:53.393 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 80/167 2024-01-16T14:57:53.447 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 81/167 2024-01-16T14:57:53.491 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 82/167 2024-01-16T14:57:53.547 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 83/167 2024-01-16T14:57:53.593 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 84/167 2024-01-16T14:57:53.726 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-01-16T14:57:53.736 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 85/167 2024-01-16T14:57:53.782 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 86/167 2024-01-16T14:57:53.884 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/167 2024-01-16T14:57:53.919 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : thrift-0.15.0-2.el9.x86_64 2/167 2024-01-16T14:57:53.923 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 87/167 2024-01-16T14:57:54.005 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 88/167 2024-01-16T14:57:54.005 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/167 2024-01-16T14:57:54.065 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 4/167 2024-01-16T14:57:54.111 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/167 2024-01-16T14:57:54.217 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/167 2024-01-16T14:57:54.611 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/167 2024-01-16T14:57:54.698 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librados2-2:18.0.0-7629.g73a4532f.el9.x86_64 8/167 2024-01-16T14:57:54.854 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-2:18.0.0-7629.g73a4532f.el9.x86_64 8/167 2024-01-16T14:57:54.876 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs2-2:18.0.0-7629.g73a4532f.el9.x86_64 9/167 2024-01-16T14:57:54.967 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 89/167 2024-01-16T14:57:54.989 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephfs2-2:18.0.0-7629.g73a4532f.el9.x86_64 9/167 2024-01-16T14:57:55.020 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 90/167 2024-01-16T14:57:55.034 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rados-2:18.0.0-7629.g73a4532f.el9.x86_64 10/167 2024-01-16T14:57:55.045 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 91/167 2024-01-16T14:57:55.070 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 92/167 2024-01-16T14:57:55.174 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/167 2024-01-16T14:57:55.318 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 12/167 2024-01-16T14:57:55.355 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/167 2024-01-16T14:57:55.383 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/167 2024-01-16T14:57:55.483 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/167 2024-01-16T14:57:55.613 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 13/167 2024-01-16T14:57:55.674 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-13.11-1.el9.x86_64 14/167 2024-01-16T14:57:55.692 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephsqlite-2:18.0.0-7629.g73a4532f.el9.x86_64 15/167 2024-01-16T14:57:55.821 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7629.g73a4532f.el9.x86_64 15/167 2024-01-16T14:57:55.843 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libradosstriper1-2:18.0.0-7629.g73a4532f.el9.x86 16/167 2024-01-16T14:57:55.919 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7629.g73a4532f.el9.x86 16/167 2024-01-16T14:57:56.212 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 17/167 2024-01-16T14:57:56.275 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/167 2024-01-16T14:57:56.399 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-libnm-1:1.45.9-1.el9.x86_64 18/167 2024-01-16T14:57:56.410 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/167 2024-01-16T14:57:56.503 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.9-1.el9.x86_64 18/167 2024-01-16T14:57:56.704 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.45.9-1.el9.x86_64 19/167 2024-01-16T14:57:56.760 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-1:1.45.9-1.el9.x86_64 19/167 2024-01-16T14:57:57.206 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.45.9-1.el9.x86_64 19/167 2024-01-16T14:57:57.257 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 20/167 2024-01-16T14:57:57.365 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/167 2024-01-16T14:57:57.440 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 22/167 2024-01-16T14:57:57.478 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 23/167 2024-01-16T14:57:57.528 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 24/167 2024-01-16T14:57:57.581 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 25/167 2024-01-16T14:57:57.711 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 26/167 2024-01-16T14:57:57.763 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 27/167 2024-01-16T14:57:57.810 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/167 2024-01-16T14:57:57.849 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lua-5.4.4-4.el9.x86_64 29/167 2024-01-16T14:57:57.965 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 30/167 2024-01-16T14:57:57.987 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/167 2024-01-16T14:57:58.064 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/167 2024-01-16T14:57:58.102 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/167 2024-01-16T14:57:58.136 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 101/167 2024-01-16T14:57:58.338 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-common-2:18.0.0-7629.g73a4532f.el9. 31/167 2024-01-16T14:57:58.410 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-argparse-2:18.0.0-7629.g73a4532f.el 32/167 2024-01-16T14:57:58.445 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cephfs-2:18.0.0-7629.g73a4532f.el9.x86_6 33/167 2024-01-16T14:57:58.483 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/167 2024-01-16T14:57:58.512 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : librbd1-2:18.0.0-7629.g73a4532f.el9.x86_64 103/167 2024-01-16T14:57:58.570 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/167 2024-01-16T14:57:58.640 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librbd1-2:18.0.0-7629.g73a4532f.el9.x86_64 103/167 2024-01-16T14:57:58.676 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rbd-2:18.0.0-7629.g73a4532f.el9.x86_64 104/167 2024-01-16T14:57:58.683 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/167 2024-01-16T14:57:58.731 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/167 2024-01-16T14:57:58.856 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/167 2024-01-16T14:57:58.938 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/167 2024-01-16T14:57:59.195 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 106/167 2024-01-16T14:57:59.232 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librgw2-2:18.0.0-7629.g73a4532f.el9.x86_64 107/167 2024-01-16T14:57:59.323 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librgw2-2:18.0.0-7629.g73a4532f.el9.x86_64 107/167 2024-01-16T14:57:59.351 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rgw-2:18.0.0-7629.g73a4532f.el9.x86_64 108/167 2024-01-16T14:57:59.834 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/167 2024-01-16T14:57:59.940 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/167 2024-01-16T14:57:59.999 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 40/167 2024-01-16T14:58:00.081 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 41/167 2024-01-16T14:58:00.131 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librados-devel-2:18.0.0-7629.g73a4532f.el9.x86_6 42/167 2024-01-16T14:58:00.204 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/167 2024-01-16T14:58:00.414 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 44/167 2024-01-16T14:58:00.456 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 45/167 2024-01-16T14:58:00.625 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : re2-1:20211101-3.el9.x86_64 46/167 2024-01-16T14:58:00.668 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 47/167 2024-01-16T14:58:00.734 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/167 2024-01-16T14:58:00.817 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 49/167 2024-01-16T14:58:00.852 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 50/167 2024-01-16T14:58:00.882 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 51/167 2024-01-16T14:58:00.937 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 52/167 2024-01-16T14:58:00.987 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 53/167 2024-01-16T14:58:01.047 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/167 2024-01-16T14:58:01.229 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/167 2024-01-16T14:58:01.275 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/167 2024-01-16T14:58:01.319 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 57/167 2024-01-16T14:58:01.456 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 58/167 2024-01-16T14:58:01.500 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-google-auth-1:2.26.0-1.el9.noarch 59/167 2024-01-16T14:58:01.535 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 60/167 2024-01-16T14:58:01.565 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 61/167 2024-01-16T14:58:01.711 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 62/167 2024-01-16T14:58:01.785 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 63/167 2024-01-16T14:58:01.839 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-common-2:18.0.0-7629.g73a4532f.el9.x86_64 109/167 2024-01-16T14:58:01.852 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ply-3.11-14.el9.noarch 64/167 2024-01-16T14:58:01.872 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-common-2:18.0.0-7629.g73a4532f.el9.x86_64 109/167 2024-01-16T14:58:01.928 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 65/167 2024-01-16T14:58:02.181 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 66/167 2024-01-16T14:58:02.252 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 67/167 2024-01-16T14:58:02.423 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 68/167 2024-01-16T14:58:02.472 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 69/167 2024-01-16T14:58:02.496 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 70/167 2024-01-16T14:58:02.507 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-common-2:18.0.0-7629.g73a4532f.el9.x86_64 109/167 2024-01-16T14:58:02.535 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 71/167 2024-01-16T14:58:02.551 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-base-2:18.0.0-7629.g73a4532f.el9.x86_64 110/167 2024-01-16T14:58:02.579 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 72/167 2024-01-16T14:58:02.624 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 73/167 2024-01-16T14:58:02.669 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : jq-1.6-16.el9.x86_64 74/167 2024-01-16T14:58:02.709 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : fuse-2.9.9-15.el9.x86_64 75/167 2024-01-16T14:58:02.712 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-base-2:18.0.0-7629.g73a4532f.el9.x86_64 110/167 2024-01-16T14:58:02.712 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-01-16T14:58:02.712 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-01-16T14:58:02.713 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:02.754 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 76/167 2024-01-16T14:58:02.761 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-selinux-2:18.0.0-7629.g73a4532f.el9.x86_64 111/167 2024-01-16T14:58:02.810 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 77/167 2024-01-16T14:58:02.874 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : uuid-1.6.2-55.el9.x86_64 78/167 2024-01-16T14:58:02.944 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 79/167 2024-01-16T14:58:03.085 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 80/167 2024-01-16T14:58:03.120 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 81/167 2024-01-16T14:58:03.164 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 82/167 2024-01-16T14:58:03.217 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 83/167 2024-01-16T14:58:03.258 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 84/167 2024-01-16T14:58:03.402 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 85/167 2024-01-16T14:58:03.449 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 86/167 2024-01-16T14:58:03.613 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 87/167 2024-01-16T14:58:03.674 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 88/167 2024-01-16T14:58:04.659 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 89/167 2024-01-16T14:58:04.717 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 90/167 2024-01-16T14:58:04.751 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 91/167 2024-01-16T14:58:04.785 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 92/167 2024-01-16T14:58:05.039 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/167 2024-01-16T14:58:05.072 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/167 2024-01-16T14:58:05.166 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/167 2024-01-16T14:58:05.982 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/167 2024-01-16T14:58:06.116 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/167 2024-01-16T14:58:07.671 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/167 2024-01-16T14:58:07.745 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/167 2024-01-16T14:58:07.782 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/167 2024-01-16T14:58:07.816 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 101/167 2024-01-16T14:58:08.172 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/167 2024-01-16T14:58:08.208 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librbd1-2:18.0.0-7629.g73a4532f.el9.x86_64 103/167 2024-01-16T14:58:08.302 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-2:18.0.0-7629.g73a4532f.el9.x86_64 103/167 2024-01-16T14:58:08.339 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rbd-2:18.0.0-7629.g73a4532f.el9.x86_64 104/167 2024-01-16T14:58:08.393 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/167 2024-01-16T14:58:08.868 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 106/167 2024-01-16T14:58:08.894 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librgw2-2:18.0.0-7629.g73a4532f.el9.x86_64 107/167 2024-01-16T14:58:08.986 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librgw2-2:18.0.0-7629.g73a4532f.el9.x86_64 107/167 2024-01-16T14:58:09.013 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rgw-2:18.0.0-7629.g73a4532f.el9.x86_64 108/167 2024-01-16T14:58:11.447 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:18.0.0-7629.g73a4532f.el9.x86_64 109/167 2024-01-16T14:58:11.485 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-common-2:18.0.0-7629.g73a4532f.el9.x86_64 109/167 2024-01-16T14:58:12.130 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:18.0.0-7629.g73a4532f.el9.x86_64 109/167 2024-01-16T14:58:12.171 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-base-2:18.0.0-7629.g73a4532f.el9.x86_64 110/167 2024-01-16T14:58:12.357 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-base-2:18.0.0-7629.g73a4532f.el9.x86_64 110/167 2024-01-16T14:58:12.357 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-01-16T14:58:12.357 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-01-16T14:58:12.357 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:12.390 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-selinux-2:18.0.0-7629.g73a4532f.el9.x86_64 111/167 2024-01-16T14:58:15.628 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7629.g73a4532f.el9.x86_64 111/167 2024-01-16T14:58:15.628 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /sys 2024-01-16T14:58:15.628 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /proc 2024-01-16T14:58:15.628 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /mnt 2024-01-16T14:58:15.628 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /var/tmp 2024-01-16T14:58:15.628 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /home 2024-01-16T14:58:15.628 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /root 2024-01-16T14:58:15.629 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /tmp 2024-01-16T14:58:15.629 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:15.848 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-volume-2:18.0.0-7629.g73a4532f.el9.noarch 112/167 2024-01-16T14:58:15.890 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-volume-2:18.0.0-7629.g73a4532f.el9.noarch 112/167 2024-01-16T14:58:15.891 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-16T14:58:15.891 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-01-16T14:58:15.891 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:17.558 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-osd-2:18.0.0-7629.g73a4532f.el9.x86_64 113/167 2024-01-16T14:58:17.653 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-osd-2:18.0.0-7629.g73a4532f.el9.x86_64 113/167 2024-01-16T14:58:17.654 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-16T14:58:17.654 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-01-16T14:58:17.654 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-16T14:58:17.654 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-16T14:58:17.654 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:17.867 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mds-2:18.0.0-7629.g73a4532f.el9.x86_64 114/167 2024-01-16T14:58:17.954 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mds-2:18.0.0-7629.g73a4532f.el9.x86_64 114/167 2024-01-16T14:58:17.955 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-16T14:58:17.955 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-01-16T14:58:17.955 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-16T14:58:17.955 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-16T14:58:17.955 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:18.459 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mon-2:18.0.0-7629.g73a4532f.el9.x86_64 115/167 2024-01-16T14:58:18.537 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mon-2:18.0.0-7629.g73a4532f.el9.x86_64 115/167 2024-01-16T14:58:18.537 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-16T14:58:18.537 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-01-16T14:58:18.537 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-16T14:58:18.538 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-16T14:58:18.538 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:18.678 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : automake-1.16.2-8.el9.noarch 116/167 2024-01-16T14:58:19.241 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: cephadm-2:18.0.0-7629.g73a4532f.el9.noarch 117/167 2024-01-16T14:58:19.278 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : cephadm-2:18.0.0-7629.g73a4532f.el9.noarch 117/167 2024-01-16T14:58:19.333 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7629.g73a4532f.e 118/167 2024-01-16T14:58:19.428 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7629.g73a4532f. 119/167 2024-01-16T14:58:19.458 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7629.g73a4532f.el9.noa 120/167 2024-01-16T14:58:19.774 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7629.g73a4532f.el9.noa 120/167 2024-01-16T14:58:19.826 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7629.g73a4532f.el9.n 121/167 2024-01-16T14:58:20.922 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7629.g73a4532f.el9.n 121/167 2024-01-16T14:58:20.967 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7629.g73a 122/167 2024-01-16T14:58:21.107 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7629.g73a 122/167 2024-01-16T14:58:21.291 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7629.g73a4532f.el 123/167 2024-01-16T14:58:21.320 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-2:18.0.0-7629.g73a4532f.el9.x86_64 124/167 2024-01-16T14:58:21.387 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7629.g73a4532f.el9.x86_64 124/167 2024-01-16T14:58:21.387 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-16T14:58:21.388 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-01-16T14:58:21.388 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-16T14:58:21.388 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-16T14:58:21.388 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:21.445 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-rook-2:18.0.0-7629.g73a4532f.el9.noarch 125/167 2024-01-16T14:58:21.483 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7629.g73a4532f.el9.noarch 125/167 2024-01-16T14:58:21.593 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-2:18.0.0-7629.g73a4532f.el9.x86_64 126/167 2024-01-16T14:58:22.499 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libtool-2.4.6-45.el9.x86_64 127/167 2024-01-16T14:58:22.531 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-radosgw-2:18.0.0-7629.g73a4532f.el9.x86_64 128/167 2024-01-16T14:58:22.607 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7629.g73a4532f.el9.x86_64 128/167 2024-01-16T14:58:22.607 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-16T14:58:22.607 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-01-16T14:58:22.607 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-16T14:58:22.607 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-16T14:58:22.607 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:22.697 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7629.g73a45 129/167 2024-01-16T14:58:22.766 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7629.g73a45 129/167 2024-01-16T14:58:22.766 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-16T14:58:22.766 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-01-16T14:58:22.766 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:22.841 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : cephfs-mirror-2:18.0.0-7629.g73a4532f.el9.x86_64 130/167 2024-01-16T14:58:22.913 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7629.g73a4532f.el9.x86_64 130/167 2024-01-16T14:58:22.913 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-16T14:58:22.913 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-01-16T14:58:22.913 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-01-16T14:58:22.914 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-01-16T14:58:22.914 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:23.283 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : rbd-mirror-2:18.0.0-7629.g73a4532f.el9.x86_64 131/167 2024-01-16T14:58:23.372 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7629.g73a4532f.el9.x86_64 131/167 2024-01-16T14:58:23.372 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-16T14:58:23.372 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-01-16T14:58:23.372 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-16T14:58:23.372 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-16T14:58:23.372 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:25.017 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7629.g73a4532f.el9.x86_64 111/167 2024-01-16T14:58:25.017 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /sys 2024-01-16T14:58:25.018 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /proc 2024-01-16T14:58:25.018 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /mnt 2024-01-16T14:58:25.018 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /var/tmp 2024-01-16T14:58:25.018 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /home 2024-01-16T14:58:25.018 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /root 2024-01-16T14:58:25.018 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /tmp 2024-01-16T14:58:25.018 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:25.238 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-volume-2:18.0.0-7629.g73a4532f.el9.noarch 112/167 2024-01-16T14:58:25.287 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-volume-2:18.0.0-7629.g73a4532f.el9.noarch 112/167 2024-01-16T14:58:25.287 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-16T14:58:25.287 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-01-16T14:58:25.287 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:26.961 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-osd-2:18.0.0-7629.g73a4532f.el9.x86_64 113/167 2024-01-16T14:58:27.044 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-osd-2:18.0.0-7629.g73a4532f.el9.x86_64 113/167 2024-01-16T14:58:27.045 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-16T14:58:27.045 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-01-16T14:58:27.045 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-16T14:58:27.045 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-16T14:58:27.045 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:27.261 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mds-2:18.0.0-7629.g73a4532f.el9.x86_64 114/167 2024-01-16T14:58:27.333 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mds-2:18.0.0-7629.g73a4532f.el9.x86_64 114/167 2024-01-16T14:58:27.333 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-16T14:58:27.334 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-01-16T14:58:27.334 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-16T14:58:27.334 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-16T14:58:27.334 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:27.471 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-test-2:18.0.0-7629.g73a4532f.el9.x86_64 132/167 2024-01-16T14:58:27.517 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : rbd-fuse-2:18.0.0-7629.g73a4532f.el9.x86_64 133/167 2024-01-16T14:58:27.572 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : rbd-nbd-2:18.0.0-7629.g73a4532f.el9.x86_64 134/167 2024-01-16T14:58:27.604 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 135/167 2024-01-16T14:58:27.820 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mon-2:18.0.0-7629.g73a4532f.el9.x86_64 115/167 2024-01-16T14:58:27.871 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 136/167 2024-01-16T14:58:27.897 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mon-2:18.0.0-7629.g73a4532f.el9.x86_64 115/167 2024-01-16T14:58:27.897 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-16T14:58:27.897 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-01-16T14:58:27.898 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-16T14:58:27.898 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-16T14:58:27.898 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:27.996 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 137/167 2024-01-16T14:58:28.005 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : automake-1.16.2-8.el9.noarch 116/167 2024-01-16T14:58:28.034 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-fuse-2:18.0.0-7629.g73a4532f.el9.x86_64 138/167 2024-01-16T14:58:28.067 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libcephfs-devel-2:18.0.0-7629.g73a4532f.el9.x86_ 139/167 2024-01-16T14:58:28.115 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 140/167 2024-01-16T14:58:28.161 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : NetworkManager-team-1:1.45.9-1.el9.x86_64 141/167 2024-01-16T14:58:28.283 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : NetworkManager-tui-1:1.45.9-1.el9.x86_64 142/167 2024-01-16T14:58:28.344 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 143/167 2024-01-16T14:58:28.386 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 144/167 2024-01-16T14:58:28.623 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephadm-2:18.0.0-7629.g73a4532f.el9.noarch 117/167 2024-01-16T14:58:28.664 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephadm-2:18.0.0-7629.g73a4532f.el9.noarch 117/167 2024-01-16T14:58:28.700 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7629.g73a4532f.e 118/167 2024-01-16T14:58:28.785 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7629.g73a4532f. 119/167 2024-01-16T14:58:28.819 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7629.g73a4532f.el9.noa 120/167 2024-01-16T14:58:29.131 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7629.g73a4532f.el9.noa 120/167 2024-01-16T14:58:29.169 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7629.g73a4532f.el9.n 121/167 2024-01-16T14:58:29.319 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 145/167 2024-01-16T14:58:29.360 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 145/167 2024-01-16T14:58:29.452 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 145/167 2024-01-16T14:58:29.537 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 146/167 2024-01-16T14:58:30.294 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7629.g73a4532f.el9.n 121/167 2024-01-16T14:58:30.320 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7629.g73a 122/167 2024-01-16T14:58:30.462 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7629.g73a 122/167 2024-01-16T14:58:30.627 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7629.g73a4532f.el 123/167 2024-01-16T14:58:30.656 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-2:18.0.0-7629.g73a4532f.el9.x86_64 124/167 2024-01-16T14:58:30.728 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7629.g73a4532f.el9.x86_64 124/167 2024-01-16T14:58:30.728 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-16T14:58:30.728 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-01-16T14:58:30.728 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-16T14:58:30.728 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-16T14:58:30.728 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:30.781 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-rook-2:18.0.0-7629.g73a4532f.el9.noarch 125/167 2024-01-16T14:58:30.817 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7629.g73a4532f.el9.noarch 125/167 2024-01-16T14:58:30.866 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-2:18.0.0-7629.g73a4532f.el9.x86_64 126/167 2024-01-16T14:58:30.873 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : cephfs-top-2:18.0.0-7629.g73a4532f.el9.noarch 147/167 2024-01-16T14:58:30.958 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 148/167 2024-01-16T14:58:31.006 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 149/167 2024-01-16T14:58:31.139 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : flex-2.6.4-9.el9.x86_64 150/167 2024-01-16T14:58:31.177 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : bison-3.7.4-5.el9.x86_64 151/167 2024-01-16T14:58:31.211 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 152/167 2024-01-16T14:58:31.245 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 153/167 2024-01-16T14:58:31.287 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 154/167 2024-01-16T14:58:31.287 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 155/167 2024-01-16T14:58:31.419 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 155/167 2024-01-16T14:58:31.462 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 156/167 2024-01-16T14:58:31.509 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 157/167 2024-01-16T14:58:31.549 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 158/167 2024-01-16T14:58:31.583 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 159/167 2024-01-16T14:58:31.583 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/167 2024-01-16T14:58:31.720 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/167 2024-01-16T14:58:31.730 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libtool-2.4.6-45.el9.x86_64 127/167 2024-01-16T14:58:31.742 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 161/167 2024-01-16T14:58:31.766 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-radosgw-2:18.0.0-7629.g73a4532f.el9.x86_64 128/167 2024-01-16T14:58:31.779 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 162/167 2024-01-16T14:58:31.779 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 162/167 2024-01-16T14:58:31.833 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7629.g73a4532f.el9.x86_64 128/167 2024-01-16T14:58:31.833 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-16T14:58:31.833 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-01-16T14:58:31.833 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-16T14:58:31.833 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-16T14:58:31.833 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:31.884 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7629.g73a45 129/167 2024-01-16T14:58:31.968 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7629.g73a45 129/167 2024-01-16T14:58:31.968 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-16T14:58:31.968 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-01-16T14:58:31.968 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:32.043 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-mirror-2:18.0.0-7629.g73a4532f.el9.x86_64 130/167 2024-01-16T14:58:32.101 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 162/167 2024-01-16T14:58:32.101 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 163/167 2024-01-16T14:58:32.139 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7629.g73a4532f.el9.x86_64 130/167 2024-01-16T14:58:32.139 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-16T14:58:32.139 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-01-16T14:58:32.139 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-01-16T14:58:32.139 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-01-16T14:58:32.139 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:32.234 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 163/167 2024-01-16T14:58:32.270 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 164/167 2024-01-16T14:58:32.309 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 165/167 2024-01-16T14:58:32.363 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 166/167 2024-01-16T14:58:32.492 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 167/167 2024-01-16T14:58:32.518 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-mirror-2:18.0.0-7629.g73a4532f.el9.x86_64 131/167 2024-01-16T14:58:32.599 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7629.g73a4532f.el9.x86_64 131/167 2024-01-16T14:58:32.599 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-16T14:58:32.599 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-01-16T14:58:32.599 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-16T14:58:32.599 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-16T14:58:32.600 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:32.745 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 167/167 2024-01-16T14:58:32.745 INFO:teuthology.orchestra.run.smithi143.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-01-16T14:58:32.745 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:36.774 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-test-2:18.0.0-7629.g73a4532f.el9.x86_64 132/167 2024-01-16T14:58:36.820 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-fuse-2:18.0.0-7629.g73a4532f.el9.x86_64 133/167 2024-01-16T14:58:36.875 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-nbd-2:18.0.0-7629.g73a4532f.el9.x86_64 134/167 2024-01-16T14:58:36.906 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 135/167 2024-01-16T14:58:37.157 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 136/167 2024-01-16T14:58:37.273 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 137/167 2024-01-16T14:58:37.310 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-fuse-2:18.0.0-7629.g73a4532f.el9.x86_64 138/167 2024-01-16T14:58:37.344 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs-devel-2:18.0.0-7629.g73a4532f.el9.x86_ 139/167 2024-01-16T14:58:37.375 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 140/167 2024-01-16T14:58:37.400 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-2:18.0.0-7629.g73a4532f.el9.x86_64 1/167 2024-01-16T14:58:37.400 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-base-2:18.0.0-7629.g73a4532f.el9.x86_64 2/167 2024-01-16T14:58:37.400 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-common-2:18.0.0-7629.g73a4532f.el9.x86_64 3/167 2024-01-16T14:58:37.400 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-fuse-2:18.0.0-7629.g73a4532f.el9.x86_64 4/167 2024-01-16T14:58:37.400 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7629.g73a45 5/167 2024-01-16T14:58:37.400 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mds-2:18.0.0-7629.g73a4532f.el9.x86_64 6/167 2024-01-16T14:58:37.400 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-2:18.0.0-7629.g73a4532f.el9.x86_64 7/167 2024-01-16T14:58:37.400 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mon-2:18.0.0-7629.g73a4532f.el9.x86_64 8/167 2024-01-16T14:58:37.400 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-osd-2:18.0.0-7629.g73a4532f.el9.x86_64 9/167 2024-01-16T14:58:37.401 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-radosgw-2:18.0.0-7629.g73a4532f.el9.x86_64 10/167 2024-01-16T14:58:37.401 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-selinux-2:18.0.0-7629.g73a4532f.el9.x86_64 11/167 2024-01-16T14:58:37.401 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-test-2:18.0.0-7629.g73a4532f.el9.x86_64 12/167 2024-01-16T14:58:37.401 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : cephfs-mirror-2:18.0.0-7629.g73a4532f.el9.x86_64 13/167 2024-01-16T14:58:37.401 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libcephfs-devel-2:18.0.0-7629.g73a4532f.el9.x86_ 14/167 2024-01-16T14:58:37.401 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libcephfs2-2:18.0.0-7629.g73a4532f.el9.x86_64 15/167 2024-01-16T14:58:37.401 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libcephsqlite-2:18.0.0-7629.g73a4532f.el9.x86_64 16/167 2024-01-16T14:58:37.401 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librados-devel-2:18.0.0-7629.g73a4532f.el9.x86_6 17/167 2024-01-16T14:58:37.401 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libradosstriper1-2:18.0.0-7629.g73a4532f.el9.x86 18/167 2024-01-16T14:58:37.401 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librgw2-2:18.0.0-7629.g73a4532f.el9.x86_64 19/167 2024-01-16T14:58:37.401 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7629.g73a4532f.el 20/167 2024-01-16T14:58:37.401 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-ceph-common-2:18.0.0-7629.g73a4532f.el9. 21/167 2024-01-16T14:58:37.401 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cephfs-2:18.0.0-7629.g73a4532f.el9.x86_6 22/167 2024-01-16T14:58:37.402 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rados-2:18.0.0-7629.g73a4532f.el9.x86_64 23/167 2024-01-16T14:58:37.402 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rbd-2:18.0.0-7629.g73a4532f.el9.x86_64 24/167 2024-01-16T14:58:37.402 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rgw-2:18.0.0-7629.g73a4532f.el9.x86_64 25/167 2024-01-16T14:58:37.415 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : rbd-fuse-2:18.0.0-7629.g73a4532f.el9.x86_64 26/167 2024-01-16T14:58:37.415 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : rbd-mirror-2:18.0.0-7629.g73a4532f.el9.x86_64 27/167 2024-01-16T14:58:37.415 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : rbd-nbd-2:18.0.0-7629.g73a4532f.el9.x86_64 28/167 2024-01-16T14:58:37.415 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7629.g73a4532f. 29/167 2024-01-16T14:58:37.415 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7629.g73a4532f.el9.noa 30/167 2024-01-16T14:58:37.415 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7629.g73a4532f.el9.n 31/167 2024-01-16T14:58:37.415 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7629.g73a 32/167 2024-01-16T14:58:37.415 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7629.g73a4532f.el 33/167 2024-01-16T14:58:37.415 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7629.g73a4532f.el9.noarch 34/167 2024-01-16T14:58:37.415 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7629.g73a4532f.e 35/167 2024-01-16T14:58:37.415 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-volume-2:18.0.0-7629.g73a4532f.el9.noarch 36/167 2024-01-16T14:58:37.416 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : cephadm-2:18.0.0-7629.g73a4532f.el9.noarch 37/167 2024-01-16T14:58:37.416 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : cephfs-top-2:18.0.0-7629.g73a4532f.el9.noarch 38/167 2024-01-16T14:58:37.416 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/167 2024-01-16T14:58:37.416 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/167 2024-01-16T14:58:37.416 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/167 2024-01-16T14:58:37.416 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 42/167 2024-01-16T14:58:37.416 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/167 2024-01-16T14:58:37.416 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/167 2024-01-16T14:58:37.416 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/167 2024-01-16T14:58:37.416 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/167 2024-01-16T14:58:37.416 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/167 2024-01-16T14:58:37.416 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/167 2024-01-16T14:58:37.416 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/167 2024-01-16T14:58:37.416 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/167 2024-01-16T14:58:37.417 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/167 2024-01-16T14:58:37.417 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 52/167 2024-01-16T14:58:37.417 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 53/167 2024-01-16T14:58:37.417 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 54/167 2024-01-16T14:58:37.417 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : lua-5.4.4-4.el9.x86_64 55/167 2024-01-16T14:58:37.417 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/167 2024-01-16T14:58:37.417 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/167 2024-01-16T14:58:37.417 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 58/167 2024-01-16T14:58:37.417 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : postgresql-13.11-1.el9.x86_64 59/167 2024-01-16T14:58:37.417 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 60/167 2024-01-16T14:58:37.417 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 61/167 2024-01-16T14:58:37.417 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 62/167 2024-01-16T14:58:37.417 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 63/167 2024-01-16T14:58:37.417 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 64/167 2024-01-16T14:58:37.418 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/167 2024-01-16T14:58:37.418 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/167 2024-01-16T14:58:37.418 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/167 2024-01-16T14:58:37.418 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/167 2024-01-16T14:58:37.418 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/167 2024-01-16T14:58:37.418 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/167 2024-01-16T14:58:37.418 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/167 2024-01-16T14:58:37.418 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 72/167 2024-01-16T14:58:37.418 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 73/167 2024-01-16T14:58:37.418 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 74/167 2024-01-16T14:58:37.418 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 75/167 2024-01-16T14:58:37.418 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 76/167 2024-01-16T14:58:37.418 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 77/167 2024-01-16T14:58:37.418 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 78/167 2024-01-16T14:58:37.418 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 79/167 2024-01-16T14:58:37.418 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 80/167 2024-01-16T14:58:37.419 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 81/167 2024-01-16T14:58:37.419 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 82/167 2024-01-16T14:58:37.419 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 83/167 2024-01-16T14:58:37.419 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 84/167 2024-01-16T14:58:37.419 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : jq-1.6-16.el9.x86_64 85/167 2024-01-16T14:58:37.419 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 86/167 2024-01-16T14:58:37.419 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 87/167 2024-01-16T14:58:37.419 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 88/167 2024-01-16T14:58:37.419 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 89/167 2024-01-16T14:58:37.419 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 90/167 2024-01-16T14:58:37.419 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-ply-3.11-14.el9.noarch 91/167 2024-01-16T14:58:37.419 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 92/167 2024-01-16T14:58:37.419 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 93/167 2024-01-16T14:58:37.419 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/167 2024-01-16T14:58:37.419 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 95/167 2024-01-16T14:58:37.420 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 96/167 2024-01-16T14:58:37.420 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 97/167 2024-01-16T14:58:37.420 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 98/167 2024-01-16T14:58:37.420 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 99/167 2024-01-16T14:58:37.420 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 100/167 2024-01-16T14:58:37.420 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 101/167 2024-01-16T14:58:37.420 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 102/167 2024-01-16T14:58:37.420 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 103/167 2024-01-16T14:58:37.420 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 104/167 2024-01-16T14:58:37.420 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 105/167 2024-01-16T14:58:37.420 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 106/167 2024-01-16T14:58:37.420 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 107/167 2024-01-16T14:58:37.420 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 108/167 2024-01-16T14:58:37.421 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 109/167 2024-01-16T14:58:37.421 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-google-auth-1:2.26.0-1.el9.noarch 110/167 2024-01-16T14:58:37.421 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 111/167 2024-01-16T14:58:37.421 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 112/167 2024-01-16T14:58:37.421 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 113/167 2024-01-16T14:58:37.421 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 114/167 2024-01-16T14:58:37.421 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 115/167 2024-01-16T14:58:37.421 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 116/167 2024-01-16T14:58:37.421 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 117/167 2024-01-16T14:58:37.421 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 118/167 2024-01-16T14:58:37.421 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 119/167 2024-01-16T14:58:37.421 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 120/167 2024-01-16T14:58:37.421 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 121/167 2024-01-16T14:58:37.421 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 122/167 2024-01-16T14:58:37.422 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 123/167 2024-01-16T14:58:37.422 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 124/167 2024-01-16T14:58:37.422 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 125/167 2024-01-16T14:58:37.422 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 126/167 2024-01-16T14:58:37.422 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 127/167 2024-01-16T14:58:37.422 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 128/167 2024-01-16T14:58:37.422 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 129/167 2024-01-16T14:58:37.422 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 130/167 2024-01-16T14:58:37.422 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 131/167 2024-01-16T14:58:37.422 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 132/167 2024-01-16T14:58:37.422 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 133/167 2024-01-16T14:58:37.422 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : re2-1:20211101-3.el9.x86_64 134/167 2024-01-16T14:58:37.422 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 135/167 2024-01-16T14:58:37.422 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librados2-2:18.0.0-7629.g73a4532f.el9.x86_64 136/167 2024-01-16T14:58:37.423 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 137/167 2024-01-16T14:58:37.423 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librbd1-2:18.0.0-7629.g73a4532f.el9.x86_64 138/167 2024-01-16T14:58:37.423 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 139/167 2024-01-16T14:58:37.423 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 140/167 2024-01-16T14:58:37.423 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 141/167 2024-01-16T14:58:37.423 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-1:1.45.9-1.el9.x86_64 142/167 2024-01-16T14:58:37.423 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 143/167 2024-01-16T14:58:37.423 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-libnm-1:1.45.9-1.el9.x86_64 144/167 2024-01-16T14:58:37.423 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 145/167 2024-01-16T14:58:37.423 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-team-1:1.45.9-1.el9.x86_64 146/167 2024-01-16T14:58:37.423 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 147/167 2024-01-16T14:58:37.423 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-tui-1:1.45.9-1.el9.x86_64 148/167 2024-01-16T14:58:37.423 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 149/167 2024-01-16T14:58:37.424 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : acl-2.3.1-4.el9.x86_64 150/167 2024-01-16T14:58:37.424 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : acl-2.3.1-3.el9.x86_64 151/167 2024-01-16T14:58:37.424 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 152/167 2024-01-16T14:58:37.424 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 153/167 2024-01-16T14:58:37.424 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 154/167 2024-01-16T14:58:37.424 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 155/167 2024-01-16T14:58:37.424 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 156/167 2024-01-16T14:58:37.424 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 157/167 2024-01-16T14:58:37.424 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 158/167 2024-01-16T14:58:37.424 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 159/167 2024-01-16T14:58:37.424 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 160/167 2024-01-16T14:58:37.424 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 161/167 2024-01-16T14:58:37.425 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 162/167 2024-01-16T14:58:37.425 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 163/167 2024-01-16T14:58:37.425 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 164/167 2024-01-16T14:58:37.425 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 165/167 2024-01-16T14:58:37.425 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 166/167 2024-01-16T14:58:37.425 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-team-1:1.45.9-1.el9.x86_64 141/167 2024-01-16T14:58:37.538 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-tui-1:1.45.9-1.el9.x86_64 142/167 2024-01-16T14:58:37.612 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 143/167 2024-01-16T14:58:37.654 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 144/167 2024-01-16T14:58:38.491 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 145/167 2024-01-16T14:58:38.536 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 145/167 2024-01-16T14:58:38.620 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 145/167 2024-01-16T14:58:38.680 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 146/167 2024-01-16T14:58:39.634 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 167/167 2024-01-16T14:58:39.634 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:39.635 INFO:teuthology.orchestra.run.smithi143.stdout:Upgraded: 2024-01-16T14:58:39.635 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-1:1.45.9-1.el9.x86_64 2024-01-16T14:58:39.635 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-libnm-1:1.45.9-1.el9.x86_64 2024-01-16T14:58:39.635 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-team-1:1.45.9-1.el9.x86_64 2024-01-16T14:58:39.635 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-tui-1:1.45.9-1.el9.x86_64 2024-01-16T14:58:39.635 INFO:teuthology.orchestra.run.smithi143.stdout: acl-2.3.1-4.el9.x86_64 2024-01-16T14:58:39.635 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-01-16T14:58:39.635 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-01-16T14:58:39.635 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-01-16T14:58:39.635 INFO:teuthology.orchestra.run.smithi143.stdout: libacl-2.3.1-4.el9.x86_64 2024-01-16T14:58:39.635 INFO:teuthology.orchestra.run.smithi143.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-01-16T14:58:39.635 INFO:teuthology.orchestra.run.smithi143.stdout: librados2-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:39.635 INFO:teuthology.orchestra.run.smithi143.stdout: librbd1-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:39.635 INFO:teuthology.orchestra.run.smithi143.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-01-16T14:58:39.635 INFO:teuthology.orchestra.run.smithi143.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-01-16T14:58:39.636 INFO:teuthology.orchestra.run.smithi143.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-01-16T14:58:39.636 INFO:teuthology.orchestra.run.smithi143.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-01-16T14:58:39.636 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2024-01-16T14:58:39.636 INFO:teuthology.orchestra.run.smithi143.stdout: automake-1.16.2-8.el9.noarch 2024-01-16T14:58:39.636 INFO:teuthology.orchestra.run.smithi143.stdout: bison-3.7.4-5.el9.x86_64 2024-01-16T14:58:39.636 INFO:teuthology.orchestra.run.smithi143.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-01-16T14:58:39.636 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:39.636 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-base-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:39.636 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-common-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:39.636 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-fuse-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:39.636 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-grafana-dashboards-2:18.0.0-7629.g73a4532f.el9.noarch 2024-01-16T14:58:39.636 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-immutable-object-cache-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:39.636 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mds-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:39.637 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:39.637 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-cephadm-2:18.0.0-7629.g73a4532f.el9.noarch 2024-01-16T14:58:39.637 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-dashboard-2:18.0.0-7629.g73a4532f.el9.noarch 2024-01-16T14:58:39.637 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7629.g73a4532f.el9.noarch 2024-01-16T14:58:39.637 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-modules-core-2:18.0.0-7629.g73a4532f.el9.noarch 2024-01-16T14:58:39.637 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-rook-2:18.0.0-7629.g73a4532f.el9.noarch 2024-01-16T14:58:39.637 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mon-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:39.637 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-osd-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:39.637 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-prometheus-alerts-2:18.0.0-7629.g73a4532f.el9.noarch 2024-01-16T14:58:39.639 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-radosgw-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:39.639 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-selinux-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:39.639 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-test-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:39.639 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-volume-2:18.0.0-7629.g73a4532f.el9.noarch 2024-01-16T14:58:39.639 INFO:teuthology.orchestra.run.smithi143.stdout: cephadm-2:18.0.0-7629.g73a4532f.el9.noarch 2024-01-16T14:58:39.639 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-mirror-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:39.639 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-top-2:18.0.0-7629.g73a4532f.el9.noarch 2024-01-16T14:58:39.639 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-01-16T14:58:39.640 INFO:teuthology.orchestra.run.smithi143.stdout: flex-2.6.4-9.el9.x86_64 2024-01-16T14:58:39.640 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-01-16T14:58:39.640 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-01-16T14:58:39.640 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-01-16T14:58:39.640 INFO:teuthology.orchestra.run.smithi143.stdout: fuse-2.9.9-15.el9.x86_64 2024-01-16T14:58:39.640 INFO:teuthology.orchestra.run.smithi143.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-01-16T14:58:39.640 INFO:teuthology.orchestra.run.smithi143.stdout: jq-1.6-16.el9.x86_64 2024-01-16T14:58:39.640 INFO:teuthology.orchestra.run.smithi143.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-01-16T14:58:39.640 INFO:teuthology.orchestra.run.smithi143.stdout: libarrow-9.0.0-10.el9.x86_64 2024-01-16T14:58:39.640 INFO:teuthology.orchestra.run.smithi143.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-01-16T14:58:39.640 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs-devel-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:39.640 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs2-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:39.640 INFO:teuthology.orchestra.run.smithi143.stdout: libcephsqlite-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:39.640 INFO:teuthology.orchestra.run.smithi143.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-01-16T14:58:39.641 INFO:teuthology.orchestra.run.smithi143.stdout: liboath-2.6.7-2.el9.x86_64 2024-01-16T14:58:39.641 INFO:teuthology.orchestra.run.smithi143.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-01-16T14:58:39.641 INFO:teuthology.orchestra.run.smithi143.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-01-16T14:58:39.641 INFO:teuthology.orchestra.run.smithi143.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-01-16T14:58:39.641 INFO:teuthology.orchestra.run.smithi143.stdout: librados-devel-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:39.641 INFO:teuthology.orchestra.run.smithi143.stdout: libradosstriper1-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:39.641 INFO:teuthology.orchestra.run.smithi143.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-01-16T14:58:39.641 INFO:teuthology.orchestra.run.smithi143.stdout: librgw2-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:39.641 INFO:teuthology.orchestra.run.smithi143.stdout: libtool-2.4.6-45.el9.x86_64 2024-01-16T14:58:39.641 INFO:teuthology.orchestra.run.smithi143.stdout: libunwind-1.6.2-1.el9.x86_64 2024-01-16T14:58:39.641 INFO:teuthology.orchestra.run.smithi143.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2024-01-16T14:58:39.641 INFO:teuthology.orchestra.run.smithi143.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-01-16T14:58:39.641 INFO:teuthology.orchestra.run.smithi143.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-01-16T14:58:39.641 INFO:teuthology.orchestra.run.smithi143.stdout: lua-5.4.4-4.el9.x86_64 2024-01-16T14:58:39.641 INFO:teuthology.orchestra.run.smithi143.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-01-16T14:58:39.641 INFO:teuthology.orchestra.run.smithi143.stdout: luarocks-3.9.2-1.el9.noarch 2024-01-16T14:58:39.642 INFO:teuthology.orchestra.run.smithi143.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-01-16T14:58:39.642 INFO:teuthology.orchestra.run.smithi143.stdout: openblas-0.3.21-2.el9.x86_64 2024-01-16T14:58:39.642 INFO:teuthology.orchestra.run.smithi143.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-01-16T14:58:39.642 INFO:teuthology.orchestra.run.smithi143.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-01-16T14:58:39.642 INFO:teuthology.orchestra.run.smithi143.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-01-16T14:58:39.642 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-13.11-1.el9.x86_64 2024-01-16T14:58:39.642 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-01-16T14:58:39.642 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-01-16T14:58:39.642 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-server-13.11-1.el9.x86_64 2024-01-16T14:58:39.642 INFO:teuthology.orchestra.run.smithi143.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-01-16T14:58:39.642 INFO:teuthology.orchestra.run.smithi143.stdout: python3-babel-2.9.1-2.el9.noarch 2024-01-16T14:58:39.642 INFO:teuthology.orchestra.run.smithi143.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-01-16T14:58:39.642 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-01-16T14:58:39.642 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-argparse-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:39.642 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-common-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:39.643 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cephfs-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:39.643 INFO:teuthology.orchestra.run.smithi143.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-01-16T14:58:39.643 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-01-16T14:58:39.643 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-01-16T14:58:39.643 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-01-16T14:58:39.643 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-01-16T14:58:39.643 INFO:teuthology.orchestra.run.smithi143.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-01-16T14:58:39.643 INFO:teuthology.orchestra.run.smithi143.stdout: python3-google-auth-1:2.26.0-1.el9.noarch 2024-01-16T14:58:39.643 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-01-16T14:58:39.643 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-01-16T14:58:39.643 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-01-16T14:58:39.643 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-01-16T14:58:39.643 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-01-16T14:58:39.643 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jinja2-2.11.3-4.el9.noarch 2024-01-16T14:58:39.643 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-01-16T14:58:39.644 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-01-16T14:58:39.644 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-01-16T14:58:39.644 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-01-16T14:58:39.644 INFO:teuthology.orchestra.run.smithi143.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-01-16T14:58:39.644 INFO:teuthology.orchestra.run.smithi143.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-01-16T14:58:39.644 INFO:teuthology.orchestra.run.smithi143.stdout: python3-mako-1.1.4-6.el9.noarch 2024-01-16T14:58:39.644 INFO:teuthology.orchestra.run.smithi143.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-01-16T14:58:39.644 INFO:teuthology.orchestra.run.smithi143.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-01-16T14:58:39.644 INFO:teuthology.orchestra.run.smithi143.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-01-16T14:58:39.644 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-01-16T14:58:39.644 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-01-16T14:58:39.644 INFO:teuthology.orchestra.run.smithi143.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-01-16T14:58:39.644 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-01-16T14:58:39.644 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ply-3.11-14.el9.noarch 2024-01-16T14:58:39.645 INFO:teuthology.orchestra.run.smithi143.stdout: python3-portend-3.1.0-2.el9.noarch 2024-01-16T14:58:39.645 INFO:teuthology.orchestra.run.smithi143.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-01-16T14:58:39.645 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-01-16T14:58:39.645 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-01-16T14:58:39.645 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-01-16T14:58:39.645 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pycparser-2.20-6.el9.noarch 2024-01-16T14:58:39.645 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pytz-2021.1-5.el9.noarch 2024-01-16T14:58:39.645 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rados-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:39.645 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rbd-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:39.645 INFO:teuthology.orchestra.run.smithi143.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-01-16T14:58:39.645 INFO:teuthology.orchestra.run.smithi143.stdout: python3-requests-2.25.1-7.el9.noarch 2024-01-16T14:58:39.645 INFO:teuthology.orchestra.run.smithi143.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-01-16T14:58:39.645 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rgw-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:39.645 INFO:teuthology.orchestra.run.smithi143.stdout: python3-routes-2.5.1-5.el9.noarch 2024-01-16T14:58:39.645 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rsa-4.9-2.el9.noarch 2024-01-16T14:58:39.645 INFO:teuthology.orchestra.run.smithi143.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-01-16T14:58:39.646 INFO:teuthology.orchestra.run.smithi143.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-01-16T14:58:39.646 INFO:teuthology.orchestra.run.smithi143.stdout: python3-toml-0.10.2-6.el9.noarch 2024-01-16T14:58:39.646 INFO:teuthology.orchestra.run.smithi143.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-01-16T14:58:39.646 INFO:teuthology.orchestra.run.smithi143.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-01-16T14:58:39.646 INFO:teuthology.orchestra.run.smithi143.stdout: python3-webob-1.8.7-6.el9.noarch 2024-01-16T14:58:39.646 INFO:teuthology.orchestra.run.smithi143.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-01-16T14:58:39.646 INFO:teuthology.orchestra.run.smithi143.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-01-16T14:58:39.646 INFO:teuthology.orchestra.run.smithi143.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-01-16T14:58:39.646 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-fuse-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:39.646 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-mirror-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:39.646 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-nbd-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:39.646 INFO:teuthology.orchestra.run.smithi143.stdout: re2-1:20211101-3.el9.x86_64 2024-01-16T14:58:39.646 INFO:teuthology.orchestra.run.smithi143.stdout: socat-1.7.4.1-5.el9.x86_64 2024-01-16T14:58:39.646 INFO:teuthology.orchestra.run.smithi143.stdout: thrift-0.15.0-2.el9.x86_64 2024-01-16T14:58:39.646 INFO:teuthology.orchestra.run.smithi143.stdout: uuid-1.6.2-55.el9.x86_64 2024-01-16T14:58:39.647 INFO:teuthology.orchestra.run.smithi143.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-01-16T14:58:39.647 INFO:teuthology.orchestra.run.smithi143.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-01-16T14:58:39.647 INFO:teuthology.orchestra.run.smithi143.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-01-16T14:58:39.647 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:39.647 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-01-16T14:58:40.018 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-top-2:18.0.0-7629.g73a4532f.el9.noarch 147/167 2024-01-16T14:58:40.092 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 148/167 2024-01-16T14:58:40.135 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 149/167 2024-01-16T14:58:40.229 DEBUG:teuthology.parallel:result is None 2024-01-16T14:58:40.282 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flex-2.6.4-9.el9.x86_64 150/167 2024-01-16T14:58:40.320 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : bison-3.7.4-5.el9.x86_64 151/167 2024-01-16T14:58:40.353 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 152/167 2024-01-16T14:58:40.387 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 153/167 2024-01-16T14:58:40.428 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 154/167 2024-01-16T14:58:40.428 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 155/167 2024-01-16T14:58:40.553 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 155/167 2024-01-16T14:58:40.603 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 156/167 2024-01-16T14:58:40.651 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 157/167 2024-01-16T14:58:40.691 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 158/167 2024-01-16T14:58:40.747 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 159/167 2024-01-16T14:58:40.748 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/167 2024-01-16T14:58:40.879 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/167 2024-01-16T14:58:40.909 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 161/167 2024-01-16T14:58:40.947 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 162/167 2024-01-16T14:58:40.948 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 162/167 2024-01-16T14:58:41.276 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 162/167 2024-01-16T14:58:41.276 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 163/167 2024-01-16T14:58:41.418 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 163/167 2024-01-16T14:58:41.454 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 164/167 2024-01-16T14:58:41.493 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 165/167 2024-01-16T14:58:41.539 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 166/167 2024-01-16T14:58:41.648 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 167/167 2024-01-16T14:58:41.911 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 167/167 2024-01-16T14:58:41.911 INFO:teuthology.orchestra.run.smithi204.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-01-16T14:58:41.911 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:46.832 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-2:18.0.0-7629.g73a4532f.el9.x86_64 1/167 2024-01-16T14:58:46.833 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-base-2:18.0.0-7629.g73a4532f.el9.x86_64 2/167 2024-01-16T14:58:46.833 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-common-2:18.0.0-7629.g73a4532f.el9.x86_64 3/167 2024-01-16T14:58:46.833 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-fuse-2:18.0.0-7629.g73a4532f.el9.x86_64 4/167 2024-01-16T14:58:46.833 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7629.g73a45 5/167 2024-01-16T14:58:46.833 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mds-2:18.0.0-7629.g73a4532f.el9.x86_64 6/167 2024-01-16T14:58:46.833 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-2:18.0.0-7629.g73a4532f.el9.x86_64 7/167 2024-01-16T14:58:46.833 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mon-2:18.0.0-7629.g73a4532f.el9.x86_64 8/167 2024-01-16T14:58:46.833 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-osd-2:18.0.0-7629.g73a4532f.el9.x86_64 9/167 2024-01-16T14:58:46.833 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-radosgw-2:18.0.0-7629.g73a4532f.el9.x86_64 10/167 2024-01-16T14:58:46.833 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-selinux-2:18.0.0-7629.g73a4532f.el9.x86_64 11/167 2024-01-16T14:58:46.833 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-test-2:18.0.0-7629.g73a4532f.el9.x86_64 12/167 2024-01-16T14:58:46.833 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-mirror-2:18.0.0-7629.g73a4532f.el9.x86_64 13/167 2024-01-16T14:58:46.834 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs-devel-2:18.0.0-7629.g73a4532f.el9.x86_ 14/167 2024-01-16T14:58:46.834 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs2-2:18.0.0-7629.g73a4532f.el9.x86_64 15/167 2024-01-16T14:58:46.834 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephsqlite-2:18.0.0-7629.g73a4532f.el9.x86_64 16/167 2024-01-16T14:58:46.834 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados-devel-2:18.0.0-7629.g73a4532f.el9.x86_6 17/167 2024-01-16T14:58:46.834 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libradosstriper1-2:18.0.0-7629.g73a4532f.el9.x86 18/167 2024-01-16T14:58:46.834 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librgw2-2:18.0.0-7629.g73a4532f.el9.x86_64 19/167 2024-01-16T14:58:46.834 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7629.g73a4532f.el 20/167 2024-01-16T14:58:46.834 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-common-2:18.0.0-7629.g73a4532f.el9. 21/167 2024-01-16T14:58:46.834 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cephfs-2:18.0.0-7629.g73a4532f.el9.x86_6 22/167 2024-01-16T14:58:46.834 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rados-2:18.0.0-7629.g73a4532f.el9.x86_64 23/167 2024-01-16T14:58:46.834 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rbd-2:18.0.0-7629.g73a4532f.el9.x86_64 24/167 2024-01-16T14:58:46.834 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rgw-2:18.0.0-7629.g73a4532f.el9.x86_64 25/167 2024-01-16T14:58:46.834 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-fuse-2:18.0.0-7629.g73a4532f.el9.x86_64 26/167 2024-01-16T14:58:46.834 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-mirror-2:18.0.0-7629.g73a4532f.el9.x86_64 27/167 2024-01-16T14:58:46.835 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-nbd-2:18.0.0-7629.g73a4532f.el9.x86_64 28/167 2024-01-16T14:58:46.835 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7629.g73a4532f. 29/167 2024-01-16T14:58:46.835 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7629.g73a4532f.el9.noa 30/167 2024-01-16T14:58:46.835 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7629.g73a4532f.el9.n 31/167 2024-01-16T14:58:46.835 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7629.g73a 32/167 2024-01-16T14:58:46.835 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7629.g73a4532f.el 33/167 2024-01-16T14:58:46.835 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7629.g73a4532f.el9.noarch 34/167 2024-01-16T14:58:46.835 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7629.g73a4532f.e 35/167 2024-01-16T14:58:46.835 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-volume-2:18.0.0-7629.g73a4532f.el9.noarch 36/167 2024-01-16T14:58:46.835 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephadm-2:18.0.0-7629.g73a4532f.el9.noarch 37/167 2024-01-16T14:58:46.835 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-top-2:18.0.0-7629.g73a4532f.el9.noarch 38/167 2024-01-16T14:58:46.835 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/167 2024-01-16T14:58:46.835 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/167 2024-01-16T14:58:46.835 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/167 2024-01-16T14:58:46.836 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 42/167 2024-01-16T14:58:46.836 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/167 2024-01-16T14:58:46.836 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/167 2024-01-16T14:58:46.836 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/167 2024-01-16T14:58:46.836 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/167 2024-01-16T14:58:46.836 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/167 2024-01-16T14:58:46.836 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/167 2024-01-16T14:58:46.836 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/167 2024-01-16T14:58:46.836 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/167 2024-01-16T14:58:46.836 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/167 2024-01-16T14:58:46.836 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 52/167 2024-01-16T14:58:46.836 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 53/167 2024-01-16T14:58:46.836 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 54/167 2024-01-16T14:58:46.836 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lua-5.4.4-4.el9.x86_64 55/167 2024-01-16T14:58:46.837 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/167 2024-01-16T14:58:46.837 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/167 2024-01-16T14:58:46.837 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 58/167 2024-01-16T14:58:46.837 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-13.11-1.el9.x86_64 59/167 2024-01-16T14:58:46.837 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 60/167 2024-01-16T14:58:46.837 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 61/167 2024-01-16T14:58:46.837 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 62/167 2024-01-16T14:58:46.837 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 63/167 2024-01-16T14:58:46.837 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 64/167 2024-01-16T14:58:46.837 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/167 2024-01-16T14:58:46.837 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/167 2024-01-16T14:58:46.837 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/167 2024-01-16T14:58:46.837 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/167 2024-01-16T14:58:46.837 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/167 2024-01-16T14:58:46.837 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/167 2024-01-16T14:58:46.837 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/167 2024-01-16T14:58:46.838 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 72/167 2024-01-16T14:58:46.838 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 73/167 2024-01-16T14:58:46.838 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 74/167 2024-01-16T14:58:46.838 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 75/167 2024-01-16T14:58:46.838 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 76/167 2024-01-16T14:58:46.838 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 77/167 2024-01-16T14:58:46.838 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 78/167 2024-01-16T14:58:46.838 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 79/167 2024-01-16T14:58:46.838 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 80/167 2024-01-16T14:58:46.838 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 81/167 2024-01-16T14:58:46.838 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 82/167 2024-01-16T14:58:46.838 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 83/167 2024-01-16T14:58:46.838 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 84/167 2024-01-16T14:58:46.838 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : jq-1.6-16.el9.x86_64 85/167 2024-01-16T14:58:46.839 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 86/167 2024-01-16T14:58:46.839 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 87/167 2024-01-16T14:58:46.839 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 88/167 2024-01-16T14:58:46.839 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 89/167 2024-01-16T14:58:46.839 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 90/167 2024-01-16T14:58:46.839 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ply-3.11-14.el9.noarch 91/167 2024-01-16T14:58:46.839 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 92/167 2024-01-16T14:58:46.839 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 93/167 2024-01-16T14:58:46.839 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/167 2024-01-16T14:58:46.839 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 95/167 2024-01-16T14:58:46.839 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 96/167 2024-01-16T14:58:46.839 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 97/167 2024-01-16T14:58:46.839 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 98/167 2024-01-16T14:58:46.839 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 99/167 2024-01-16T14:58:46.839 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 100/167 2024-01-16T14:58:46.839 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 101/167 2024-01-16T14:58:46.840 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 102/167 2024-01-16T14:58:46.840 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 103/167 2024-01-16T14:58:46.840 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 104/167 2024-01-16T14:58:46.840 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 105/167 2024-01-16T14:58:46.840 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 106/167 2024-01-16T14:58:46.840 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 107/167 2024-01-16T14:58:46.840 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 108/167 2024-01-16T14:58:46.840 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 109/167 2024-01-16T14:58:46.840 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-google-auth-1:2.26.0-1.el9.noarch 110/167 2024-01-16T14:58:46.840 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 111/167 2024-01-16T14:58:46.840 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 112/167 2024-01-16T14:58:46.840 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 113/167 2024-01-16T14:58:46.841 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 114/167 2024-01-16T14:58:46.841 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 115/167 2024-01-16T14:58:46.841 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 116/167 2024-01-16T14:58:46.841 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 117/167 2024-01-16T14:58:46.841 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 118/167 2024-01-16T14:58:46.841 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 119/167 2024-01-16T14:58:46.841 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 120/167 2024-01-16T14:58:46.841 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 121/167 2024-01-16T14:58:46.841 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 122/167 2024-01-16T14:58:46.841 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 123/167 2024-01-16T14:58:46.841 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 124/167 2024-01-16T14:58:46.841 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 125/167 2024-01-16T14:58:46.841 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 126/167 2024-01-16T14:58:46.841 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 127/167 2024-01-16T14:58:46.841 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 128/167 2024-01-16T14:58:46.841 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 129/167 2024-01-16T14:58:46.842 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 130/167 2024-01-16T14:58:46.842 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 131/167 2024-01-16T14:58:46.842 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 132/167 2024-01-16T14:58:46.842 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 133/167 2024-01-16T14:58:46.842 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : re2-1:20211101-3.el9.x86_64 134/167 2024-01-16T14:58:46.842 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 135/167 2024-01-16T14:58:46.842 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-2:18.0.0-7629.g73a4532f.el9.x86_64 136/167 2024-01-16T14:58:46.842 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 137/167 2024-01-16T14:58:46.842 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-2:18.0.0-7629.g73a4532f.el9.x86_64 138/167 2024-01-16T14:58:46.842 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 139/167 2024-01-16T14:58:46.842 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 140/167 2024-01-16T14:58:46.842 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 141/167 2024-01-16T14:58:46.842 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-1:1.45.9-1.el9.x86_64 142/167 2024-01-16T14:58:46.842 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 143/167 2024-01-16T14:58:46.843 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-libnm-1:1.45.9-1.el9.x86_64 144/167 2024-01-16T14:58:46.843 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 145/167 2024-01-16T14:58:46.843 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-team-1:1.45.9-1.el9.x86_64 146/167 2024-01-16T14:58:46.843 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 147/167 2024-01-16T14:58:46.843 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-tui-1:1.45.9-1.el9.x86_64 148/167 2024-01-16T14:58:46.843 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 149/167 2024-01-16T14:58:46.843 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : acl-2.3.1-4.el9.x86_64 150/167 2024-01-16T14:58:46.843 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : acl-2.3.1-3.el9.x86_64 151/167 2024-01-16T14:58:46.843 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 152/167 2024-01-16T14:58:46.843 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 153/167 2024-01-16T14:58:46.843 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 154/167 2024-01-16T14:58:46.843 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 155/167 2024-01-16T14:58:46.843 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 156/167 2024-01-16T14:58:46.843 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 157/167 2024-01-16T14:58:46.844 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 158/167 2024-01-16T14:58:46.844 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 159/167 2024-01-16T14:58:46.844 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 160/167 2024-01-16T14:58:46.844 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 161/167 2024-01-16T14:58:46.844 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 162/167 2024-01-16T14:58:46.844 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 163/167 2024-01-16T14:58:46.844 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 164/167 2024-01-16T14:58:46.844 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 165/167 2024-01-16T14:58:46.844 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 166/167 2024-01-16T14:58:49.074 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 167/167 2024-01-16T14:58:49.074 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:49.074 INFO:teuthology.orchestra.run.smithi204.stdout:Upgraded: 2024-01-16T14:58:49.074 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-1:1.45.9-1.el9.x86_64 2024-01-16T14:58:49.075 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-libnm-1:1.45.9-1.el9.x86_64 2024-01-16T14:58:49.075 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-team-1:1.45.9-1.el9.x86_64 2024-01-16T14:58:49.075 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-tui-1:1.45.9-1.el9.x86_64 2024-01-16T14:58:49.075 INFO:teuthology.orchestra.run.smithi204.stdout: acl-2.3.1-4.el9.x86_64 2024-01-16T14:58:49.075 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-01-16T14:58:49.075 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-01-16T14:58:49.075 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-01-16T14:58:49.075 INFO:teuthology.orchestra.run.smithi204.stdout: libacl-2.3.1-4.el9.x86_64 2024-01-16T14:58:49.075 INFO:teuthology.orchestra.run.smithi204.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-01-16T14:58:49.075 INFO:teuthology.orchestra.run.smithi204.stdout: librados2-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:49.075 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:49.075 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-01-16T14:58:49.075 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-01-16T14:58:49.076 INFO:teuthology.orchestra.run.smithi204.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-01-16T14:58:49.076 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-01-16T14:58:49.076 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-01-16T14:58:49.076 INFO:teuthology.orchestra.run.smithi204.stdout: automake-1.16.2-8.el9.noarch 2024-01-16T14:58:49.076 INFO:teuthology.orchestra.run.smithi204.stdout: bison-3.7.4-5.el9.x86_64 2024-01-16T14:58:49.076 INFO:teuthology.orchestra.run.smithi204.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-01-16T14:58:49.076 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:49.076 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:49.076 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:49.076 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:49.076 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards-2:18.0.0-7629.g73a4532f.el9.noarch 2024-01-16T14:58:49.076 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:49.077 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:49.077 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:49.077 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm-2:18.0.0-7629.g73a4532f.el9.noarch 2024-01-16T14:58:49.077 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard-2:18.0.0-7629.g73a4532f.el9.noarch 2024-01-16T14:58:49.077 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7629.g73a4532f.el9.noarch 2024-01-16T14:58:49.077 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core-2:18.0.0-7629.g73a4532f.el9.noarch 2024-01-16T14:58:49.077 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook-2:18.0.0-7629.g73a4532f.el9.noarch 2024-01-16T14:58:49.077 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:49.077 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:49.077 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts-2:18.0.0-7629.g73a4532f.el9.noarch 2024-01-16T14:58:49.077 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:49.077 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:49.077 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:49.077 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume-2:18.0.0-7629.g73a4532f.el9.noarch 2024-01-16T14:58:49.078 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm-2:18.0.0-7629.g73a4532f.el9.noarch 2024-01-16T14:58:49.078 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-mirror-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:49.078 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-top-2:18.0.0-7629.g73a4532f.el9.noarch 2024-01-16T14:58:49.078 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-01-16T14:58:49.078 INFO:teuthology.orchestra.run.smithi204.stdout: flex-2.6.4-9.el9.x86_64 2024-01-16T14:58:49.078 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-01-16T14:58:49.078 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-01-16T14:58:49.078 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-01-16T14:58:49.079 INFO:teuthology.orchestra.run.smithi204.stdout: fuse-2.9.9-15.el9.x86_64 2024-01-16T14:58:49.079 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-01-16T14:58:49.079 INFO:teuthology.orchestra.run.smithi204.stdout: jq-1.6-16.el9.x86_64 2024-01-16T14:58:49.079 INFO:teuthology.orchestra.run.smithi204.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-01-16T14:58:49.079 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-9.0.0-10.el9.x86_64 2024-01-16T14:58:49.079 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-01-16T14:58:49.079 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:49.079 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:49.079 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:49.079 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-01-16T14:58:49.079 INFO:teuthology.orchestra.run.smithi204.stdout: liboath-2.6.7-2.el9.x86_64 2024-01-16T14:58:49.079 INFO:teuthology.orchestra.run.smithi204.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-01-16T14:58:49.080 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-01-16T14:58:49.080 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-01-16T14:58:49.080 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:49.080 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:49.080 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-01-16T14:58:49.080 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:49.080 INFO:teuthology.orchestra.run.smithi204.stdout: libtool-2.4.6-45.el9.x86_64 2024-01-16T14:58:49.081 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind-1.6.2-1.el9.x86_64 2024-01-16T14:58:49.081 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2024-01-16T14:58:49.081 INFO:teuthology.orchestra.run.smithi204.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-01-16T14:58:49.081 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-01-16T14:58:49.081 INFO:teuthology.orchestra.run.smithi204.stdout: lua-5.4.4-4.el9.x86_64 2024-01-16T14:58:49.081 INFO:teuthology.orchestra.run.smithi204.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-01-16T14:58:49.081 INFO:teuthology.orchestra.run.smithi204.stdout: luarocks-3.9.2-1.el9.noarch 2024-01-16T14:58:49.081 INFO:teuthology.orchestra.run.smithi204.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-01-16T14:58:49.081 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-0.3.21-2.el9.x86_64 2024-01-16T14:58:49.081 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-01-16T14:58:49.082 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-01-16T14:58:49.082 INFO:teuthology.orchestra.run.smithi204.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-01-16T14:58:49.082 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-13.11-1.el9.x86_64 2024-01-16T14:58:49.082 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-01-16T14:58:49.082 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-01-16T14:58:49.082 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-server-13.11-1.el9.x86_64 2024-01-16T14:58:49.082 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-01-16T14:58:49.082 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel-2.9.1-2.el9.noarch 2024-01-16T14:58:49.082 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-01-16T14:58:49.082 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-01-16T14:58:49.082 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:49.082 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:49.082 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:49.082 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-01-16T14:58:49.083 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-01-16T14:58:49.083 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-01-16T14:58:49.083 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-01-16T14:58:49.083 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-01-16T14:58:49.083 INFO:teuthology.orchestra.run.smithi204.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-01-16T14:58:49.083 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth-1:2.26.0-1.el9.noarch 2024-01-16T14:58:49.083 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-01-16T14:58:49.083 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-01-16T14:58:49.083 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-01-16T14:58:49.083 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-01-16T14:58:49.083 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-01-16T14:58:49.083 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2-2.11.3-4.el9.noarch 2024-01-16T14:58:49.083 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-01-16T14:58:49.084 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-01-16T14:58:49.084 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-01-16T14:58:49.084 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-01-16T14:58:49.084 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-01-16T14:58:49.084 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-01-16T14:58:49.084 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako-1.1.4-6.el9.noarch 2024-01-16T14:58:49.084 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-01-16T14:58:49.084 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-01-16T14:58:49.084 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-01-16T14:58:49.084 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-01-16T14:58:49.084 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-01-16T14:58:49.084 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-01-16T14:58:49.085 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-01-16T14:58:49.085 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ply-3.11-14.el9.noarch 2024-01-16T14:58:49.085 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend-3.1.0-2.el9.noarch 2024-01-16T14:58:49.085 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-01-16T14:58:49.085 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-01-16T14:58:49.085 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-01-16T14:58:49.085 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-01-16T14:58:49.085 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser-2.20-6.el9.noarch 2024-01-16T14:58:49.085 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz-2021.1-5.el9.noarch 2024-01-16T14:58:49.085 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:49.085 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:49.085 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-01-16T14:58:49.085 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-2.25.1-7.el9.noarch 2024-01-16T14:58:49.086 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-01-16T14:58:49.086 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:49.086 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes-2.5.1-5.el9.noarch 2024-01-16T14:58:49.086 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa-4.9-2.el9.noarch 2024-01-16T14:58:49.086 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-01-16T14:58:49.086 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-01-16T14:58:49.086 INFO:teuthology.orchestra.run.smithi204.stdout: python3-toml-0.10.2-6.el9.noarch 2024-01-16T14:58:49.086 INFO:teuthology.orchestra.run.smithi204.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-01-16T14:58:49.086 INFO:teuthology.orchestra.run.smithi204.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-01-16T14:58:49.086 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob-1.8.7-6.el9.noarch 2024-01-16T14:58:49.086 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-01-16T14:58:49.086 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-01-16T14:58:49.086 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-01-16T14:58:49.086 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:49.086 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:49.087 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd-2:18.0.0-7629.g73a4532f.el9.x86_64 2024-01-16T14:58:49.087 INFO:teuthology.orchestra.run.smithi204.stdout: re2-1:20211101-3.el9.x86_64 2024-01-16T14:58:49.087 INFO:teuthology.orchestra.run.smithi204.stdout: socat-1.7.4.1-5.el9.x86_64 2024-01-16T14:58:49.087 INFO:teuthology.orchestra.run.smithi204.stdout: thrift-0.15.0-2.el9.x86_64 2024-01-16T14:58:49.087 INFO:teuthology.orchestra.run.smithi204.stdout: uuid-1.6.2-55.el9.x86_64 2024-01-16T14:58:49.087 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-01-16T14:58:49.087 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-01-16T14:58:49.088 INFO:teuthology.orchestra.run.smithi204.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-01-16T14:58:49.088 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:49.088 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-01-16T14:58:49.715 DEBUG:teuthology.parallel:result is None 2024-01-16T14:58:49.716 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=73a4532fc4d97adee0b985d4d42874ea351a1c86 2024-01-16T14:58:49.864 DEBUG:teuthology.orchestra.run.smithi143:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-01-16T14:58:49.936 INFO:teuthology.orchestra.run.smithi143.stdout:18.0.0-7629.g73a4532f.el9 2024-01-16T14:58:49.937 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7629.g73a4532f.el9 2024-01-16T14:58:49.937 INFO:teuthology.task.install:The correct ceph version 18.0.0-7629.g73a4532f is installed. 2024-01-16T14:58:49.938 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=73a4532fc4d97adee0b985d4d42874ea351a1c86 2024-01-16T14:58:50.081 DEBUG:teuthology.orchestra.run.smithi204:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-01-16T14:58:50.137 INFO:teuthology.orchestra.run.smithi204.stdout:18.0.0-7629.g73a4532f.el9 2024-01-16T14:58:50.138 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7629.g73a4532f.el9 2024-01-16T14:58:50.138 INFO:teuthology.task.install:The correct ceph version 18.0.0-7629.g73a4532f is installed. 2024-01-16T14:58:50.140 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-01-16T14:58:50.140 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T14:58:50.141 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-01-16T14:58:50.242 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-16T14:58:50.242 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-01-16T14:58:50.321 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-01-16T14:58:50.322 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T14:58:50.322 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/daemon-helper 2024-01-16T14:58:50.419 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-01-16T14:58:50.508 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-16T14:58:50.508 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/daemon-helper 2024-01-16T14:58:50.583 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-01-16T14:58:50.705 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-01-16T14:58:50.705 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T14:58:50.705 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/adjust-ulimits 2024-01-16T14:58:50.802 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-01-16T14:58:50.935 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-16T14:58:50.935 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/adjust-ulimits 2024-01-16T14:58:51.015 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-01-16T14:58:51.128 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-01-16T14:58:51.128 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T14:58:51.128 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/stdin-killer 2024-01-16T14:58:51.226 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-01-16T14:58:51.318 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-16T14:58:51.319 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/stdin-killer 2024-01-16T14:58:51.398 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-01-16T14:58:51.514 INFO:teuthology.run_tasks:Running task ceph... 2024-01-16T14:58:51.628 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-01-16T14:58:51.629 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 777 /var/log/ceph 2024-01-16T14:58:51.636 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 777 /var/log/ceph 2024-01-16T14:58:51.728 INFO:tasks.ceph:Disabling ceph logrotate... 2024-01-16T14:58:51.728 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f -- /etc/logrotate.d/ceph 2024-01-16T14:58:51.734 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /etc/logrotate.d/ceph 2024-01-16T14:58:51.822 INFO:tasks.ceph:Creating extra log directories... 2024-01-16T14:58:51.823 DEBUG:teuthology.orchestra.run.smithi143:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-01-16T14:58:51.829 DEBUG:teuthology.orchestra.run.smithi204:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-01-16T14:58:51.939 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-01-16T14:58:51.939 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-01-16T14:58:51.939 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-01-16T14:58:51.939 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T14:58:51.939 DEBUG:teuthology.orchestra.run.smithi143:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-01-16T14:58:51.990 DEBUG:teuthology.orchestra.run.smithi143:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-01-16T14:58:52.250 DEBUG:teuthology.orchestra.run.smithi143:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-01-16T14:58:52.350 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-16T14:58:52.350 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-01-16T14:58:52.395 DEBUG:teuthology.orchestra.run.smithi204:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-01-16T14:58:52.602 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-01-16T14:58:52.683 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-01-16T14:58:52.683 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-01-16T14:58:52.683 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-01-16_14:37:50-fs:volumes-rishabh-clone-stats-testing-default-smithi/7518192', 'branch': 'rishabh-clone-stats', 'description': 'fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone-progress}}', 'email': None, 'first_in_suite': False, 'job_id': '7518192', 'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '2668949e0971849893a3fb7f177ce9032fc65780'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2024-01-16_14:37:50-fs:volumes-rishabh-clone-stats-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'rishabh-clone-stats'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '73a4532fc4d97adee0b985d4d42874ea351a1c86'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '73a4532fc4d97adee0b985d4d42874ea351a1c86'}}, 'kclient': {'syntax': 'v1'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'rishabh-clone-stats', 'sha1': '73a4532fc4d97adee0b985d4d42874ea351a1c86'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 2668, 'sha1': '73a4532fc4d97adee0b985d4d42874ea351a1c86', 'sleep_before_teardown': 0, 'suite': 'fs:volumes', 'suite_branch': 'rishabh-clone-stats', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_73a4532fc4d97adee0b985d4d42874ea351a1c86/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '73a4532fc4d97adee0b985d4d42874ea351a1c86', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi143.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMEZGZXmIlZiJR1TY0oddUL9M4YS37kACpY0PZj/hAK3rR3Rgn57TOa0oCsfsM1VaszdgAJjPcjoM25mb26r/gg=', 'smithi204.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDZizvbZvwbqCGGr+goAs4E6aM8VEU0pKCgbz9zOce6zdfM6kFS6mZWPBiywmG0ueu1N6AZFIvZ0tRGoXNgBgAY='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '2668949e0971849893a3fb7f177ce9032fc65780'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '73a4532fc4d97adee0b985d4d42874ea351a1c86'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '73a4532fc4d97adee0b985d4d42874ea351a1c86', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestCloneProgress']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'cd45576300487d997e5a85abed65500b9f5d143b', 'timestamp': '2024-01-16_14:37:50', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-01-16T14:58:52.684 DEBUG:teuthology.orchestra.run.smithi143:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-01-16T14:58:52.691 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-01-16T14:58:52.740 DEBUG:teuthology.orchestra.run.smithi143:> sudo install -d -m0777 -- /var/run/ceph 2024-01-16T14:58:52.745 DEBUG:teuthology.orchestra.run.smithi204:> sudo install -d -m0777 -- /var/run/ceph 2024-01-16T14:58:52.841 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T14:58:52.841 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/scratch_devs of=/dev/stdout 2024-01-16T14:58:52.896 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-01-16T14:58:52.896 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_1 2024-01-16T14:58:52.948 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-01-16T14:58:52.948 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-16T14:58:52.948 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 5h/5d Inode: 634 Links: 1 2024-01-16T14:58:52.948 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-16T14:58:52.948 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2024-01-16T14:58:52.948 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2024-01-16 14:58:33.171646960 +0000 2024-01-16T14:58:52.948 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2024-01-16 14:56:11.341911415 +0000 2024-01-16T14:58:52.948 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2024-01-16 14:56:11.341911415 +0000 2024-01-16T14:58:52.948 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2024-01-16 14:56:11.341911415 +0000 2024-01-16T14:58:52.948 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-01-16T14:58:53.034 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2024-01-16T14:58:53.035 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2024-01-16T14:58:53.035 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000312032 s, 1.6 MB/s 2024-01-16T14:58:53.039 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-01-16T14:58:53.092 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_2 2024-01-16T14:58:53.136 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-01-16T14:58:53.136 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-16T14:58:53.136 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 5h/5d Inode: 650 Links: 1 2024-01-16T14:58:53.137 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-16T14:58:53.137 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2024-01-16T14:58:53.137 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2024-01-16 14:58:33.172646933 +0000 2024-01-16T14:58:53.137 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2024-01-16 14:56:11.900896209 +0000 2024-01-16T14:58:53.137 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2024-01-16 14:56:11.900896209 +0000 2024-01-16T14:58:53.137 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2024-01-16 14:56:11.900896209 +0000 2024-01-16T14:58:53.137 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-01-16T14:58:53.266 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2024-01-16T14:58:53.266 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2024-01-16T14:58:53.266 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000361221 s, 1.4 MB/s 2024-01-16T14:58:53.270 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-01-16T14:58:53.323 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_3 2024-01-16T14:58:53.372 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-01-16T14:58:53.372 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-16T14:58:53.372 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 5h/5d Inode: 670 Links: 1 2024-01-16T14:58:53.373 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-16T14:58:53.373 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2024-01-16T14:58:53.373 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2024-01-16 14:58:33.172646933 +0000 2024-01-16T14:58:53.373 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2024-01-16 14:56:12.469880732 +0000 2024-01-16T14:58:53.373 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2024-01-16 14:56:12.469880732 +0000 2024-01-16T14:58:53.373 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2024-01-16 14:56:12.469880732 +0000 2024-01-16T14:58:53.373 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-01-16T14:58:53.454 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2024-01-16T14:58:53.454 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2024-01-16T14:58:53.455 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000340096 s, 1.5 MB/s 2024-01-16T14:58:53.458 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-01-16T14:58:53.511 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_4 2024-01-16T14:58:53.560 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-01-16T14:58:53.561 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-16T14:58:53.561 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 5h/5d Inode: 689 Links: 1 2024-01-16T14:58:53.561 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-16T14:58:53.561 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2024-01-16T14:58:53.561 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2024-01-16 14:58:33.173646906 +0000 2024-01-16T14:58:53.561 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2024-01-16 14:56:13.030865472 +0000 2024-01-16T14:58:53.561 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2024-01-16 14:56:13.030865472 +0000 2024-01-16T14:58:53.561 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2024-01-16 14:56:13.030865472 +0000 2024-01-16T14:58:53.561 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-01-16T14:58:53.648 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2024-01-16T14:58:53.648 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2024-01-16T14:58:53.648 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000393813 s, 1.3 MB/s 2024-01-16T14:58:53.652 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-01-16T14:58:53.708 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-16T14:58:53.709 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-16T14:58:53.709 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/scratch_devs of=/dev/stdout 2024-01-16T14:58:53.757 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-01-16T14:58:53.757 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_1 2024-01-16T14:58:53.840 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-01-16T14:58:53.840 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-16T14:58:53.840 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 633 Links: 1 2024-01-16T14:58:53.840 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-16T14:58:53.840 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-01-16T14:58:53.841 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-01-16 14:58:42.370227038 +0000 2024-01-16T14:58:53.841 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-01-16 14:56:11.501691678 +0000 2024-01-16T14:58:53.841 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-01-16 14:56:11.501691678 +0000 2024-01-16T14:58:53.841 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-01-16 14:56:11.501691678 +0000 2024-01-16T14:58:53.841 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-01-16T14:58:53.951 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-01-16T14:58:53.951 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-01-16T14:58:53.951 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000345109 s, 1.5 MB/s 2024-01-16T14:58:53.954 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-01-16T14:58:54.000 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_2 2024-01-16T14:58:54.094 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-01-16T14:58:54.095 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-16T14:58:54.095 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 652 Links: 1 2024-01-16T14:58:54.095 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-16T14:58:54.095 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-01-16T14:58:54.095 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-01-16 14:58:42.371227022 +0000 2024-01-16T14:58:54.095 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-01-16 14:56:12.064683152 +0000 2024-01-16T14:58:54.095 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-01-16 14:56:12.064683152 +0000 2024-01-16T14:58:54.095 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-01-16 14:56:12.064683152 +0000 2024-01-16T14:58:54.095 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-01-16T14:58:54.172 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-01-16T14:58:54.172 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-01-16T14:58:54.172 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000379351 s, 1.3 MB/s 2024-01-16T14:58:54.176 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-01-16T14:58:54.227 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_3 2024-01-16T14:58:54.272 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-01-16T14:58:54.272 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-16T14:58:54.272 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 669 Links: 1 2024-01-16T14:58:54.272 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-16T14:58:54.272 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-01-16T14:58:54.272 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-01-16 14:58:42.371227022 +0000 2024-01-16T14:58:54.272 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-01-16 14:56:12.631674565 +0000 2024-01-16T14:58:54.273 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-01-16 14:56:12.631674565 +0000 2024-01-16T14:58:54.273 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-01-16 14:56:12.631674565 +0000 2024-01-16T14:58:54.273 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-01-16T14:58:54.400 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-01-16T14:58:54.400 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-01-16T14:58:54.401 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000371668 s, 1.4 MB/s 2024-01-16T14:58:54.405 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-01-16T14:58:54.456 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_4 2024-01-16T14:58:54.499 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-01-16T14:58:54.499 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-16T14:58:54.499 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 691 Links: 1 2024-01-16T14:58:54.499 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-16T14:58:54.499 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-01-16T14:58:54.499 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-01-16 14:58:42.373226993 +0000 2024-01-16T14:58:54.500 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-01-16 14:56:13.209665811 +0000 2024-01-16T14:58:54.500 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-01-16 14:56:13.209665811 +0000 2024-01-16T14:58:54.500 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-01-16 14:56:13.209665811 +0000 2024-01-16T14:58:54.500 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-01-16T14:58:54.612 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-01-16T14:58:54.612 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-01-16T14:58:54.612 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000292195 s, 1.8 MB/s 2024-01-16T14:58:54.615 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-01-16T14:58:54.661 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-16T14:58:54.661 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi143.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi204.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-01-16T14:58:54.661 INFO:tasks.ceph:Generating config... 2024-01-16T14:58:54.662 INFO:tasks.ceph:[client] client mount timeout = 600 2024-01-16T14:58:54.663 INFO:tasks.ceph:[client] debug client = 20 2024-01-16T14:58:54.663 INFO:tasks.ceph:[client] debug ms = 1 2024-01-16T14:58:54.663 INFO:tasks.ceph:[client] pid file = 2024-01-16T14:58:54.663 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-01-16T14:58:54.663 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-01-16T14:58:54.663 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-01-16T14:58:54.663 INFO:tasks.ceph:[mds] debug mds = 20 2024-01-16T14:58:54.663 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-01-16T14:58:54.663 INFO:tasks.ceph:[mds] debug ms = 1 2024-01-16T14:58:54.663 INFO:tasks.ceph:[mds] mds debug frag = True 2024-01-16T14:58:54.663 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-01-16T14:58:54.663 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-01-16T14:58:54.663 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-01-16T14:58:54.663 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-01-16T14:58:54.663 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-01-16T14:58:54.664 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-01-16T14:58:54.664 INFO:tasks.ceph:[mgr] debug client = 20 2024-01-16T14:58:54.664 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-01-16T14:58:54.664 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-01-16T14:58:54.664 INFO:tasks.ceph:[mgr] debug ms = 1 2024-01-16T14:58:54.664 INFO:tasks.ceph:[mon] debug mon = 20 2024-01-16T14:58:54.664 INFO:tasks.ceph:[mon] debug ms = 1 2024-01-16T14:58:54.664 INFO:tasks.ceph:[mon] debug paxos = 20 2024-01-16T14:58:54.664 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-01-16T14:58:54.664 INFO:tasks.ceph:[osd] bdev async discard = True 2024-01-16T14:58:54.664 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-01-16T14:58:54.664 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-01-16T14:58:54.664 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-01-16T14:58:54.664 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-01-16T14:58:54.664 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-01-16T14:58:54.665 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-01-16T14:58:54.665 INFO:tasks.ceph:[osd] debug ms = 1 2024-01-16T14:58:54.665 INFO:tasks.ceph:[osd] debug osd = 20 2024-01-16T14:58:54.665 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-01-16T14:58:54.665 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-01-16T14:58:54.665 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-01-16T14:58:54.665 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-01-16T14:58:54.665 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-01-16T14:58:54.665 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-01-16T14:58:54.665 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-01-16T14:58:54.665 INFO:tasks.ceph:Setting up mon.a... 2024-01-16T14:58:54.665 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-01-16T14:58:54.808 INFO:teuthology.orchestra.run.smithi143.stdout:creating /etc/ceph/ceph.keyring 2024-01-16T14:58:54.816 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-01-16T14:58:54.945 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-01-16T14:58:55.032 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.143'), ('mon.b', '172.21.15.204'), ('mon.c', '[v2:172.21.15.204:3301,v1:172.21.15.204:6790]')] 2024-01-16T14:58:55.033 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.143,172.21.15.204,[v2:172.21.15.204:3301,v1:172.21.15.204:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-01-16T14:58:55.033 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T14:58:55.034 DEBUG:teuthology.orchestra.run.smithi143:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-01-16T14:58:55.083 DEBUG:teuthology.orchestra.run.smithi143:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.143 --add b 172.21.15.204 --addv c '[v2:172.21.15.204:3301,v1:172.21.15.204:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-01-16T14:58:55.226 INFO:teuthology.orchestra.run.smithi143.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-01-16T14:58:55.226 INFO:teuthology.orchestra.run.smithi143.stdout:monmaptool: generated fsid c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0 2024-01-16T14:58:55.226 INFO:teuthology.orchestra.run.smithi143.stdout:setting min_mon_release = pacific 2024-01-16T14:58:55.226 INFO:teuthology.orchestra.run.smithi143.stdout:epoch 0 2024-01-16T14:58:55.227 INFO:teuthology.orchestra.run.smithi143.stdout:fsid c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0 2024-01-16T14:58:55.227 INFO:teuthology.orchestra.run.smithi143.stdout:last_changed 2024-01-16T14:58:55.245982+0000 2024-01-16T14:58:55.227 INFO:teuthology.orchestra.run.smithi143.stdout:created 2024-01-16T14:58:55.245982+0000 2024-01-16T14:58:55.227 INFO:teuthology.orchestra.run.smithi143.stdout:min_mon_release 16 (pacific) 2024-01-16T14:58:55.227 INFO:teuthology.orchestra.run.smithi143.stdout:election_strategy: 1 2024-01-16T14:58:55.227 INFO:teuthology.orchestra.run.smithi143.stdout:0: [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] mon.a 2024-01-16T14:58:55.227 INFO:teuthology.orchestra.run.smithi143.stdout:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-01-16T14:58:55.227 INFO:teuthology.orchestra.run.smithi143.stdout:2: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.c 2024-01-16T14:58:55.227 INFO:teuthology.orchestra.run.smithi143.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-01-16T14:58:55.229 DEBUG:teuthology.orchestra.run.smithi143:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-01-16T14:58:55.280 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0... 2024-01-16T14:58:55.281 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-01-16T14:58:55.286 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-01-16T14:58:55.441 INFO:teuthology.orchestra.run.smithi143.stdout:[global] 2024-01-16T14:58:55.441 INFO:teuthology.orchestra.run.smithi143.stdout: chdir = "" 2024-01-16T14:58:55.441 INFO:teuthology.orchestra.run.smithi143.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-01-16T14:58:55.441 INFO:teuthology.orchestra.run.smithi143.stdout: auth supported = cephx 2024-01-16T14:58:55.441 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:55.441 INFO:teuthology.orchestra.run.smithi143.stdout: filestore xattr use omap = true 2024-01-16T14:58:55.441 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:55.442 INFO:teuthology.orchestra.run.smithi143.stdout: mon clock drift allowed = 1.000 2024-01-16T14:58:55.442 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:55.442 INFO:teuthology.orchestra.run.smithi143.stdout: osd crush chooseleaf type = 0 2024-01-16T14:58:55.442 INFO:teuthology.orchestra.run.smithi143.stdout: auth debug = true 2024-01-16T14:58:55.442 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:55.442 INFO:teuthology.orchestra.run.smithi143.stdout: ms die on old message = true 2024-01-16T14:58:55.442 INFO:teuthology.orchestra.run.smithi143.stdout: ms die on bug = true 2024-01-16T14:58:55.442 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:55.442 INFO:teuthology.orchestra.run.smithi143.stdout: mon max pg per osd = 10000 # >= luminous 2024-01-16T14:58:55.442 INFO:teuthology.orchestra.run.smithi143.stdout: mon pg warn max object skew = 0 2024-01-16T14:58:55.442 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:55.442 INFO:teuthology.orchestra.run.smithi143.stdout: # disable pg_autoscaler by default for new pools 2024-01-16T14:58:55.442 INFO:teuthology.orchestra.run.smithi143.stdout: osd_pool_default_pg_autoscale_mode = off 2024-01-16T14:58:55.442 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:55.443 INFO:teuthology.orchestra.run.smithi143.stdout: osd pool default size = 2 2024-01-16T14:58:55.443 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:55.443 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd allow primary affinity = true 2024-01-16T14:58:55.443 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd allow pg remap = true 2024-01-16T14:58:55.443 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on legacy crush tunables = false 2024-01-16T14:58:55.443 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on crush straw calc version zero = false 2024-01-16T14:58:55.443 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on no sortbitwise = false 2024-01-16T14:58:55.443 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on osd down out interval zero = false 2024-01-16T14:58:55.443 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on too few osds = false 2024-01-16T14:58:55.443 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-01-16T14:58:55.443 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_pool_no_redundancy = false 2024-01-16T14:58:55.443 INFO:teuthology.orchestra.run.smithi143.stdout: mon_allow_pool_size_one = true 2024-01-16T14:58:55.443 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:55.443 INFO:teuthology.orchestra.run.smithi143.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-01-16T14:58:55.443 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:55.444 INFO:teuthology.orchestra.run.smithi143.stdout: osd default data pool replay window = 5 2024-01-16T14:58:55.444 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:55.444 INFO:teuthology.orchestra.run.smithi143.stdout: mon allow pool delete = true 2024-01-16T14:58:55.444 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:55.444 INFO:teuthology.orchestra.run.smithi143.stdout: mon cluster log file level = debug 2024-01-16T14:58:55.444 INFO:teuthology.orchestra.run.smithi143.stdout: debug asserts on shutdown = true 2024-01-16T14:58:55.444 INFO:teuthology.orchestra.run.smithi143.stdout: mon health detail to clog = false 2024-01-16T14:58:55.444 INFO:teuthology.orchestra.run.smithi143.stdout: mon host = "172.21.15.143,172.21.15.204,[v2:172.21.15.204:3301,v1:172.21.15.204:6790]" 2024-01-16T14:58:55.444 INFO:teuthology.orchestra.run.smithi143.stdout: ms die on skipped message = False 2024-01-16T14:58:55.444 INFO:teuthology.orchestra.run.smithi143.stdout: fsid = c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0 2024-01-16T14:58:55.444 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:55.444 INFO:teuthology.orchestra.run.smithi143.stdout:[osd] 2024-01-16T14:58:55.444 INFO:teuthology.orchestra.run.smithi143.stdout: osd journal size = 100 2024-01-16T14:58:55.445 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:55.445 INFO:teuthology.orchestra.run.smithi143.stdout: osd scrub load threshold = 5.0 2024-01-16T14:58:55.445 INFO:teuthology.orchestra.run.smithi143.stdout: osd scrub max interval = 600 2024-01-16T14:58:55.445 INFO:teuthology.orchestra.run.smithi143.stdout: osd mclock profile = high_recovery_ops 2024-01-16T14:58:55.445 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:55.445 INFO:teuthology.orchestra.run.smithi143.stdout: osd recover clone overlap = true 2024-01-16T14:58:55.445 INFO:teuthology.orchestra.run.smithi143.stdout: osd recovery max chunk = 1048576 2024-01-16T14:58:55.445 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:55.445 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug shutdown = true 2024-01-16T14:58:55.445 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug op order = true 2024-01-16T14:58:55.445 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug verify stray on activate = true 2024-01-16T14:58:55.445 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug trim objects = true 2024-01-16T14:58:55.445 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:55.445 INFO:teuthology.orchestra.run.smithi143.stdout: osd open classes on start = true 2024-01-16T14:58:55.445 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug pg log writeout = true 2024-01-16T14:58:55.446 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:55.446 INFO:teuthology.orchestra.run.smithi143.stdout: osd deep scrub update digest min age = 30 2024-01-16T14:58:55.446 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:55.446 INFO:teuthology.orchestra.run.smithi143.stdout: osd map max advance = 10 2024-01-16T14:58:55.446 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:55.446 INFO:teuthology.orchestra.run.smithi143.stdout: journal zero on create = true 2024-01-16T14:58:55.446 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:55.446 INFO:teuthology.orchestra.run.smithi143.stdout: filestore ondisk finisher threads = 3 2024-01-16T14:58:55.446 INFO:teuthology.orchestra.run.smithi143.stdout: filestore apply finisher threads = 3 2024-01-16T14:58:55.446 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:55.446 INFO:teuthology.orchestra.run.smithi143.stdout: bdev debug aio = true 2024-01-16T14:58:55.446 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug misdirected ops = true 2024-01-16T14:58:55.447 INFO:teuthology.orchestra.run.smithi143.stdout: bdev async discard = True 2024-01-16T14:58:55.447 INFO:teuthology.orchestra.run.smithi143.stdout: bdev enable discard = True 2024-01-16T14:58:55.447 INFO:teuthology.orchestra.run.smithi143.stdout: bluestore allocator = bitmap 2024-01-16T14:58:55.447 INFO:teuthology.orchestra.run.smithi143.stdout: bluestore block size = 96636764160 2024-01-16T14:58:55.447 INFO:teuthology.orchestra.run.smithi143.stdout: bluestore fsck on mount = True 2024-01-16T14:58:55.447 INFO:teuthology.orchestra.run.smithi143.stdout: debug bluefs = 1/20 2024-01-16T14:58:55.447 INFO:teuthology.orchestra.run.smithi143.stdout: debug bluestore = 1/20 2024-01-16T14:58:55.447 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2024-01-16T14:58:55.447 INFO:teuthology.orchestra.run.smithi143.stdout: debug osd = 20 2024-01-16T14:58:55.447 INFO:teuthology.orchestra.run.smithi143.stdout: debug rocksdb = 4/10 2024-01-16T14:58:55.447 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd backfillfull_ratio = 0.85 2024-01-16T14:58:55.447 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd full ratio = 0.9 2024-01-16T14:58:55.447 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd nearfull ratio = 0.8 2024-01-16T14:58:55.448 INFO:teuthology.orchestra.run.smithi143.stdout: osd failsafe full ratio = 0.95 2024-01-16T14:58:55.448 INFO:teuthology.orchestra.run.smithi143.stdout: osd objectstore = bluestore 2024-01-16T14:58:55.448 INFO:teuthology.orchestra.run.smithi143.stdout: osd op complaint time = 180 2024-01-16T14:58:55.448 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:55.448 INFO:teuthology.orchestra.run.smithi143.stdout:[mgr] 2024-01-16T14:58:55.448 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2024-01-16T14:58:55.448 INFO:teuthology.orchestra.run.smithi143.stdout: debug mgr = 20 2024-01-16T14:58:55.448 INFO:teuthology.orchestra.run.smithi143.stdout: debug mon = 20 2024-01-16T14:58:55.448 INFO:teuthology.orchestra.run.smithi143.stdout: debug auth = 20 2024-01-16T14:58:55.448 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min pgs per osd = 4 2024-01-16T14:58:55.448 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min bytes per osd = 10 2024-01-16T14:58:55.448 INFO:teuthology.orchestra.run.smithi143.stdout: mgr/telemetry/nag = false 2024-01-16T14:58:55.448 INFO:teuthology.orchestra.run.smithi143.stdout: debug client = 20 2024-01-16T14:58:55.449 INFO:teuthology.orchestra.run.smithi143.stdout: debug finisher = 20 2024-01-16T14:58:55.449 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:55.449 INFO:teuthology.orchestra.run.smithi143.stdout:[mon] 2024-01-16T14:58:55.449 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2024-01-16T14:58:55.449 INFO:teuthology.orchestra.run.smithi143.stdout: debug mon = 20 2024-01-16T14:58:55.449 INFO:teuthology.orchestra.run.smithi143.stdout: debug paxos = 20 2024-01-16T14:58:55.449 INFO:teuthology.orchestra.run.smithi143.stdout: debug auth = 20 2024-01-16T14:58:55.449 INFO:teuthology.orchestra.run.smithi143.stdout: mon data avail warn = 5 2024-01-16T14:58:55.449 INFO:teuthology.orchestra.run.smithi143.stdout: mon mgr mkfs grace = 240 2024-01-16T14:58:55.449 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min pgs per osd = 4 2024-01-16T14:58:55.449 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd reporter subtree level = osd 2024-01-16T14:58:55.449 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd prime pg temp = true 2024-01-16T14:58:55.450 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min bytes per osd = 10 2024-01-16T14:58:55.450 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:55.450 INFO:teuthology.orchestra.run.smithi143.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-01-16T14:58:55.450 INFO:teuthology.orchestra.run.smithi143.stdout: auth mon ticket ttl = 660 # 11m 2024-01-16T14:58:55.450 INFO:teuthology.orchestra.run.smithi143.stdout: auth service ticket ttl = 240 # 4m 2024-01-16T14:58:55.450 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:55.450 INFO:teuthology.orchestra.run.smithi143.stdout: # don't complain about insecure global_id in the test suite 2024-01-16T14:58:55.450 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-01-16T14:58:55.450 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-01-16T14:58:55.450 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:55.450 INFO:teuthology.orchestra.run.smithi143.stdout: # 1m isn't quite enough 2024-01-16T14:58:55.450 INFO:teuthology.orchestra.run.smithi143.stdout: mon_down_mkfs_grace = 2m 2024-01-16T14:58:55.450 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:55.451 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_filestore_osds = false 2024-01-16T14:58:55.451 INFO:teuthology.orchestra.run.smithi143.stdout: mon op complaint time = 120 2024-01-16T14:58:55.451 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:58:55.451 INFO:teuthology.orchestra.run.smithi143.stdout:[client] 2024-01-16T14:58:55.451 INFO:teuthology.orchestra.run.smithi143.stdout: rgw cache enabled = true 2024-01-16T14:58:55.451 INFO:teuthology.orchestra.run.smithi143.stdout: rgw enable ops log = true 2024-01-16T14:58:55.451 INFO:teuthology.orchestra.run.smithi143.stdout: rgw enable usage log = true 2024-01-16T14:58:55.451 INFO:teuthology.orchestra.run.smithi143.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-01-16T14:58:55.451 INFO:teuthology.orchestra.run.smithi143.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-01-16T14:58:55.451 INFO:teuthology.orchestra.run.smithi143.stdout: client mount timeout = 600 2024-01-16T14:58:55.451 INFO:teuthology.orchestra.run.smithi143.stdout: debug client = 20 2024-01-16T14:58:55.451 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2024-01-16T14:58:55.451 INFO:teuthology.orchestra.run.smithi143.stdout: pid file = "" 2024-01-16T14:58:55.452 INFO:teuthology.orchestra.run.smithi143.stdout: rados mon op timeout = 900 2024-01-16T14:58:55.452 INFO:teuthology.orchestra.run.smithi143.stdout: rados osd op timeout = 900 2024-01-16T14:58:55.452 INFO:teuthology.orchestra.run.smithi143.stdout:[mon.a] 2024-01-16T14:58:55.452 INFO:teuthology.orchestra.run.smithi143.stdout:[mon.b] 2024-01-16T14:58:55.452 INFO:teuthology.orchestra.run.smithi143.stdout:[mon.c] 2024-01-16T14:58:55.452 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.a] 2024-01-16T14:58:55.452 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.c] 2024-01-16T14:58:55.452 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.b] 2024-01-16T14:58:55.452 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.d] 2024-01-16T14:58:55.452 INFO:teuthology.orchestra.run.smithi143.stdout:[mds] 2024-01-16T14:58:55.452 INFO:teuthology.orchestra.run.smithi143.stdout: debug mds = 20 2024-01-16T14:58:55.452 INFO:teuthology.orchestra.run.smithi143.stdout: debug mds balancer = 20 2024-01-16T14:58:55.453 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2024-01-16T14:58:55.453 INFO:teuthology.orchestra.run.smithi143.stdout: mds debug frag = True 2024-01-16T14:58:55.453 INFO:teuthology.orchestra.run.smithi143.stdout: mds debug scatterstat = True 2024-01-16T14:58:55.453 INFO:teuthology.orchestra.run.smithi143.stdout: mds op complaint time = 180 2024-01-16T14:58:55.453 INFO:teuthology.orchestra.run.smithi143.stdout: mds verify scatter = True 2024-01-16T14:58:55.453 INFO:teuthology.orchestra.run.smithi143.stdout: osd op complaint time = 180 2024-01-16T14:58:55.453 INFO:teuthology.orchestra.run.smithi143.stdout: rados mon op timeout = 900 2024-01-16T14:58:55.453 INFO:teuthology.orchestra.run.smithi143.stdout: rados osd op timeout = 900 2024-01-16T14:58:55.453 INFO:teuthology.orchestra.run.smithi204.stdout:[global] 2024-01-16T14:58:55.454 INFO:teuthology.orchestra.run.smithi204.stdout: chdir = "" 2024-01-16T14:58:55.454 INFO:teuthology.orchestra.run.smithi204.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-01-16T14:58:55.454 INFO:teuthology.orchestra.run.smithi204.stdout: auth supported = cephx 2024-01-16T14:58:55.454 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:55.454 INFO:teuthology.orchestra.run.smithi204.stdout: filestore xattr use omap = true 2024-01-16T14:58:55.454 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:55.454 INFO:teuthology.orchestra.run.smithi204.stdout: mon clock drift allowed = 1.000 2024-01-16T14:58:55.454 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:55.454 INFO:teuthology.orchestra.run.smithi204.stdout: osd crush chooseleaf type = 0 2024-01-16T14:58:55.454 INFO:teuthology.orchestra.run.smithi204.stdout: auth debug = true 2024-01-16T14:58:55.454 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:55.454 INFO:teuthology.orchestra.run.smithi204.stdout: ms die on old message = true 2024-01-16T14:58:55.455 INFO:teuthology.orchestra.run.smithi204.stdout: ms die on bug = true 2024-01-16T14:58:55.455 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:55.455 INFO:teuthology.orchestra.run.smithi204.stdout: mon max pg per osd = 10000 # >= luminous 2024-01-16T14:58:55.455 INFO:teuthology.orchestra.run.smithi204.stdout: mon pg warn max object skew = 0 2024-01-16T14:58:55.455 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:55.455 INFO:teuthology.orchestra.run.smithi204.stdout: # disable pg_autoscaler by default for new pools 2024-01-16T14:58:55.455 INFO:teuthology.orchestra.run.smithi204.stdout: osd_pool_default_pg_autoscale_mode = off 2024-01-16T14:58:55.455 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:55.455 INFO:teuthology.orchestra.run.smithi204.stdout: osd pool default size = 2 2024-01-16T14:58:55.455 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:55.455 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd allow primary affinity = true 2024-01-16T14:58:55.455 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd allow pg remap = true 2024-01-16T14:58:55.455 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on legacy crush tunables = false 2024-01-16T14:58:55.456 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on crush straw calc version zero = false 2024-01-16T14:58:55.456 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on no sortbitwise = false 2024-01-16T14:58:55.456 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on osd down out interval zero = false 2024-01-16T14:58:55.456 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on too few osds = false 2024-01-16T14:58:55.456 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-01-16T14:58:55.456 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_pool_no_redundancy = false 2024-01-16T14:58:55.456 INFO:teuthology.orchestra.run.smithi204.stdout: mon_allow_pool_size_one = true 2024-01-16T14:58:55.456 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:55.456 INFO:teuthology.orchestra.run.smithi204.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-01-16T14:58:55.456 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:55.456 INFO:teuthology.orchestra.run.smithi204.stdout: osd default data pool replay window = 5 2024-01-16T14:58:55.456 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:55.456 INFO:teuthology.orchestra.run.smithi204.stdout: mon allow pool delete = true 2024-01-16T14:58:55.456 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:55.457 INFO:teuthology.orchestra.run.smithi204.stdout: mon cluster log file level = debug 2024-01-16T14:58:55.457 INFO:teuthology.orchestra.run.smithi204.stdout: debug asserts on shutdown = true 2024-01-16T14:58:55.457 INFO:teuthology.orchestra.run.smithi204.stdout: mon health detail to clog = false 2024-01-16T14:58:55.457 INFO:teuthology.orchestra.run.smithi204.stdout: mon host = "172.21.15.143,172.21.15.204,[v2:172.21.15.204:3301,v1:172.21.15.204:6790]" 2024-01-16T14:58:55.457 INFO:teuthology.orchestra.run.smithi204.stdout: ms die on skipped message = False 2024-01-16T14:58:55.457 INFO:teuthology.orchestra.run.smithi204.stdout: fsid = c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0 2024-01-16T14:58:55.457 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:55.457 INFO:teuthology.orchestra.run.smithi204.stdout:[osd] 2024-01-16T14:58:55.457 INFO:teuthology.orchestra.run.smithi204.stdout: osd journal size = 100 2024-01-16T14:58:55.457 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:55.457 INFO:teuthology.orchestra.run.smithi204.stdout: osd scrub load threshold = 5.0 2024-01-16T14:58:55.457 INFO:teuthology.orchestra.run.smithi204.stdout: osd scrub max interval = 600 2024-01-16T14:58:55.457 INFO:teuthology.orchestra.run.smithi204.stdout: osd mclock profile = high_recovery_ops 2024-01-16T14:58:55.457 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:55.457 INFO:teuthology.orchestra.run.smithi204.stdout: osd recover clone overlap = true 2024-01-16T14:58:55.457 INFO:teuthology.orchestra.run.smithi204.stdout: osd recovery max chunk = 1048576 2024-01-16T14:58:55.458 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:55.458 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug shutdown = true 2024-01-16T14:58:55.458 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug op order = true 2024-01-16T14:58:55.458 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug verify stray on activate = true 2024-01-16T14:58:55.458 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug trim objects = true 2024-01-16T14:58:55.458 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:55.458 INFO:teuthology.orchestra.run.smithi204.stdout: osd open classes on start = true 2024-01-16T14:58:55.458 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug pg log writeout = true 2024-01-16T14:58:55.458 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:55.458 INFO:teuthology.orchestra.run.smithi204.stdout: osd deep scrub update digest min age = 30 2024-01-16T14:58:55.458 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:55.458 INFO:teuthology.orchestra.run.smithi204.stdout: osd map max advance = 10 2024-01-16T14:58:55.458 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:55.458 INFO:teuthology.orchestra.run.smithi204.stdout: journal zero on create = true 2024-01-16T14:58:55.458 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:55.459 INFO:teuthology.orchestra.run.smithi204.stdout: filestore ondisk finisher threads = 3 2024-01-16T14:58:55.459 INFO:teuthology.orchestra.run.smithi204.stdout: filestore apply finisher threads = 3 2024-01-16T14:58:55.459 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:55.459 INFO:teuthology.orchestra.run.smithi204.stdout: bdev debug aio = true 2024-01-16T14:58:55.459 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug misdirected ops = true 2024-01-16T14:58:55.459 INFO:teuthology.orchestra.run.smithi204.stdout: bdev async discard = True 2024-01-16T14:58:55.459 INFO:teuthology.orchestra.run.smithi204.stdout: bdev enable discard = True 2024-01-16T14:58:55.459 INFO:teuthology.orchestra.run.smithi204.stdout: bluestore allocator = bitmap 2024-01-16T14:58:55.459 INFO:teuthology.orchestra.run.smithi204.stdout: bluestore block size = 96636764160 2024-01-16T14:58:55.459 INFO:teuthology.orchestra.run.smithi204.stdout: bluestore fsck on mount = True 2024-01-16T14:58:55.459 INFO:teuthology.orchestra.run.smithi204.stdout: debug bluefs = 1/20 2024-01-16T14:58:55.459 INFO:teuthology.orchestra.run.smithi204.stdout: debug bluestore = 1/20 2024-01-16T14:58:55.459 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-01-16T14:58:55.459 INFO:teuthology.orchestra.run.smithi204.stdout: debug osd = 20 2024-01-16T14:58:55.459 INFO:teuthology.orchestra.run.smithi204.stdout: debug rocksdb = 4/10 2024-01-16T14:58:55.459 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd backfillfull_ratio = 0.85 2024-01-16T14:58:55.460 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd full ratio = 0.9 2024-01-16T14:58:55.460 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd nearfull ratio = 0.8 2024-01-16T14:58:55.460 INFO:teuthology.orchestra.run.smithi204.stdout: osd failsafe full ratio = 0.95 2024-01-16T14:58:55.460 INFO:teuthology.orchestra.run.smithi204.stdout: osd objectstore = bluestore 2024-01-16T14:58:55.460 INFO:teuthology.orchestra.run.smithi204.stdout: osd op complaint time = 180 2024-01-16T14:58:55.460 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:55.460 INFO:teuthology.orchestra.run.smithi204.stdout:[mgr] 2024-01-16T14:58:55.460 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-01-16T14:58:55.460 INFO:teuthology.orchestra.run.smithi204.stdout: debug mgr = 20 2024-01-16T14:58:55.460 INFO:teuthology.orchestra.run.smithi204.stdout: debug mon = 20 2024-01-16T14:58:55.460 INFO:teuthology.orchestra.run.smithi204.stdout: debug auth = 20 2024-01-16T14:58:55.460 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min pgs per osd = 4 2024-01-16T14:58:55.460 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min bytes per osd = 10 2024-01-16T14:58:55.460 INFO:teuthology.orchestra.run.smithi204.stdout: mgr/telemetry/nag = false 2024-01-16T14:58:55.460 INFO:teuthology.orchestra.run.smithi204.stdout: debug client = 20 2024-01-16T14:58:55.461 INFO:teuthology.orchestra.run.smithi204.stdout: debug finisher = 20 2024-01-16T14:58:55.461 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:55.461 INFO:teuthology.orchestra.run.smithi204.stdout:[mon] 2024-01-16T14:58:55.461 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-01-16T14:58:55.461 INFO:teuthology.orchestra.run.smithi204.stdout: debug mon = 20 2024-01-16T14:58:55.461 INFO:teuthology.orchestra.run.smithi204.stdout: debug paxos = 20 2024-01-16T14:58:55.461 INFO:teuthology.orchestra.run.smithi204.stdout: debug auth = 20 2024-01-16T14:58:55.461 INFO:teuthology.orchestra.run.smithi204.stdout: mon data avail warn = 5 2024-01-16T14:58:55.461 INFO:teuthology.orchestra.run.smithi204.stdout: mon mgr mkfs grace = 240 2024-01-16T14:58:55.461 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min pgs per osd = 4 2024-01-16T14:58:55.461 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd reporter subtree level = osd 2024-01-16T14:58:55.461 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd prime pg temp = true 2024-01-16T14:58:55.461 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min bytes per osd = 10 2024-01-16T14:58:55.461 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:55.461 INFO:teuthology.orchestra.run.smithi204.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-01-16T14:58:55.462 INFO:teuthology.orchestra.run.smithi204.stdout: auth mon ticket ttl = 660 # 11m 2024-01-16T14:58:55.462 INFO:teuthology.orchestra.run.smithi204.stdout: auth service ticket ttl = 240 # 4m 2024-01-16T14:58:55.462 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:55.462 INFO:teuthology.orchestra.run.smithi204.stdout: # don't complain about insecure global_id in the test suite 2024-01-16T14:58:55.462 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-01-16T14:58:55.462 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-01-16T14:58:55.462 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:55.462 INFO:teuthology.orchestra.run.smithi204.stdout: # 1m isn't quite enough 2024-01-16T14:58:55.462 INFO:teuthology.orchestra.run.smithi204.stdout: mon_down_mkfs_grace = 2m 2024-01-16T14:58:55.462 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:55.462 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_filestore_osds = false 2024-01-16T14:58:55.462 INFO:teuthology.orchestra.run.smithi204.stdout: mon op complaint time = 120 2024-01-16T14:58:55.462 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-16T14:58:55.462 INFO:teuthology.orchestra.run.smithi204.stdout:[client] 2024-01-16T14:58:55.462 INFO:teuthology.orchestra.run.smithi204.stdout: rgw cache enabled = true 2024-01-16T14:58:55.462 INFO:teuthology.orchestra.run.smithi204.stdout: rgw enable ops log = true 2024-01-16T14:58:55.463 INFO:teuthology.orchestra.run.smithi204.stdout: rgw enable usage log = true 2024-01-16T14:58:55.463 INFO:teuthology.orchestra.run.smithi204.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-01-16T14:58:55.463 INFO:teuthology.orchestra.run.smithi204.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-01-16T14:58:55.463 INFO:teuthology.orchestra.run.smithi204.stdout: client mount timeout = 600 2024-01-16T14:58:55.463 INFO:teuthology.orchestra.run.smithi204.stdout: debug client = 20 2024-01-16T14:58:55.463 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-01-16T14:58:55.463 INFO:teuthology.orchestra.run.smithi204.stdout: pid file = "" 2024-01-16T14:58:55.463 INFO:teuthology.orchestra.run.smithi204.stdout: rados mon op timeout = 900 2024-01-16T14:58:55.463 INFO:teuthology.orchestra.run.smithi204.stdout: rados osd op timeout = 900 2024-01-16T14:58:55.463 INFO:teuthology.orchestra.run.smithi204.stdout:[mon.a] 2024-01-16T14:58:55.463 INFO:teuthology.orchestra.run.smithi204.stdout:[mon.b] 2024-01-16T14:58:55.463 INFO:teuthology.orchestra.run.smithi204.stdout:[mon.c] 2024-01-16T14:58:55.463 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.a] 2024-01-16T14:58:55.463 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.c] 2024-01-16T14:58:55.463 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.b] 2024-01-16T14:58:55.464 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.d] 2024-01-16T14:58:55.464 INFO:teuthology.orchestra.run.smithi204.stdout:[mds] 2024-01-16T14:58:55.464 INFO:teuthology.orchestra.run.smithi204.stdout: debug mds = 20 2024-01-16T14:58:55.464 INFO:teuthology.orchestra.run.smithi204.stdout: debug mds balancer = 20 2024-01-16T14:58:55.464 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-01-16T14:58:55.464 INFO:teuthology.orchestra.run.smithi204.stdout: mds debug frag = True 2024-01-16T14:58:55.464 INFO:teuthology.orchestra.run.smithi204.stdout: mds debug scatterstat = True 2024-01-16T14:58:55.464 INFO:teuthology.orchestra.run.smithi204.stdout: mds op complaint time = 180 2024-01-16T14:58:55.464 INFO:teuthology.orchestra.run.smithi204.stdout: mds verify scatter = True 2024-01-16T14:58:55.464 INFO:teuthology.orchestra.run.smithi204.stdout: osd op complaint time = 180 2024-01-16T14:58:55.464 INFO:teuthology.orchestra.run.smithi204.stdout: rados mon op timeout = 900 2024-01-16T14:58:55.464 INFO:teuthology.orchestra.run.smithi204.stdout: rados osd op timeout = 900 2024-01-16T14:58:55.493 INFO:tasks.ceph:Creating admin key on mon.a... 2024-01-16T14:58:55.494 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-01-16T14:58:55.635 INFO:tasks.ceph:Copying monmap to all nodes... 2024-01-16T14:58:55.636 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T14:58:55.636 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-01-16T14:58:55.685 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T14:58:55.685 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-01-16T14:58:55.780 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi143.front.sepia.ceph.com 2024-01-16T14:58:55.780 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T14:58:55.780 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/ceph/ceph.keyring 2024-01-16T14:58:55.780 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-01-16T14:58:55.920 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T14:58:55.920 DEBUG:teuthology.orchestra.run.smithi143:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-01-16T14:58:56.018 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi204.front.sepia.ceph.com 2024-01-16T14:58:56.018 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-16T14:58:56.018 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/ceph/ceph.keyring 2024-01-16T14:58:56.018 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-01-16T14:58:56.174 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-16T14:58:56.174 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-01-16T14:58:56.270 INFO:tasks.ceph:Setting up mon nodes... 2024-01-16T14:58:56.270 INFO:tasks.ceph:Setting up mgr nodes... 2024-01-16T14:58:56.270 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-01-16T14:58:56.452 INFO:teuthology.orchestra.run.smithi143.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-01-16T14:58:56.462 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-01-16T14:58:56.644 INFO:teuthology.orchestra.run.smithi204.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-01-16T14:58:56.655 INFO:tasks.ceph:Setting up mds nodes... 2024-01-16T14:58:56.655 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-01-16T14:58:56.839 INFO:teuthology.orchestra.run.smithi143.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-01-16T14:58:56.848 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-01-16T14:58:56.943 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-01-16T14:58:57.110 INFO:teuthology.orchestra.run.smithi143.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-01-16T14:58:57.119 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-01-16T14:58:57.212 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-01-16T14:58:57.397 INFO:teuthology.orchestra.run.smithi204.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-01-16T14:58:57.406 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-01-16T14:58:57.496 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-01-16T14:58:57.680 INFO:teuthology.orchestra.run.smithi204.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-01-16T14:58:57.689 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-01-16T14:58:57.782 INFO:tasks.ceph_client:Setting up client nodes... 2024-01-16T14:58:57.782 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-01-16T14:58:57.913 INFO:teuthology.orchestra.run.smithi143.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-01-16T14:58:57.969 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-01-16T14:58:58.093 INFO:teuthology.orchestra.run.smithi143.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-01-16T14:58:58.146 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-01-16T14:58:58.274 INFO:teuthology.orchestra.run.smithi204.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-01-16T14:58:58.329 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-01-16T14:58:58.452 INFO:teuthology.orchestra.run.smithi204.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-01-16T14:58:58.507 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-01-16T14:58:58.507 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi143.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi204.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-01-16T14:58:58.507 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-01-16T14:58:58.596 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-16T14:58:58.596 INFO:tasks.ceph:role: osd.0 2024-01-16T14:58:58.596 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi143.front.sepia.ceph.com 2024-01-16T14:58:58.596 DEBUG:teuthology.orchestra.run.smithi143:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-01-16T14:58:58.684 INFO:teuthology.orchestra.run.smithi143.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-01-16T14:58:58.684 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-16T14:58:58.684 INFO:teuthology.orchestra.run.smithi143.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-16T14:58:58.684 INFO:teuthology.orchestra.run.smithi143.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-16T14:58:58.684 INFO:teuthology.orchestra.run.smithi143.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-16T14:58:58.684 INFO:teuthology.orchestra.run.smithi143.stdout: = sunit=0 swidth=0 blks 2024-01-16T14:58:58.684 INFO:teuthology.orchestra.run.smithi143.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-16T14:58:58.684 INFO:teuthology.orchestra.run.smithi143.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-16T14:58:58.684 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-16T14:58:58.684 INFO:teuthology.orchestra.run.smithi143.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-16T14:58:59.204 INFO:teuthology.orchestra.run.smithi143.stdout:Discarding blocks...Done. 2024-01-16T14:58:59.210 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi143.front.sepia.ceph.com -o noatime 2024-01-16T14:58:59.210 DEBUG:teuthology.orchestra.run.smithi143:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-01-16T14:58:59.413 DEBUG:teuthology.orchestra.run.smithi143:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-01-16T14:58:59.524 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-01-16T14:58:59.604 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-16T14:58:59.605 INFO:tasks.ceph:role: osd.1 2024-01-16T14:58:59.605 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi143.front.sepia.ceph.com 2024-01-16T14:58:59.605 DEBUG:teuthology.orchestra.run.smithi143:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-01-16T14:58:59.688 INFO:teuthology.orchestra.run.smithi143.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-01-16T14:58:59.689 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-16T14:58:59.689 INFO:teuthology.orchestra.run.smithi143.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-16T14:58:59.689 INFO:teuthology.orchestra.run.smithi143.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-16T14:58:59.689 INFO:teuthology.orchestra.run.smithi143.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-16T14:58:59.689 INFO:teuthology.orchestra.run.smithi143.stdout: = sunit=0 swidth=0 blks 2024-01-16T14:58:59.689 INFO:teuthology.orchestra.run.smithi143.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-16T14:58:59.689 INFO:teuthology.orchestra.run.smithi143.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-16T14:58:59.689 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-16T14:58:59.689 INFO:teuthology.orchestra.run.smithi143.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-16T14:59:00.167 INFO:teuthology.orchestra.run.smithi143.stdout:Discarding blocks...Done. 2024-01-16T14:59:00.173 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi143.front.sepia.ceph.com -o noatime 2024-01-16T14:59:00.173 DEBUG:teuthology.orchestra.run.smithi143:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-01-16T14:59:00.305 DEBUG:teuthology.orchestra.run.smithi143:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-01-16T14:59:00.396 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-01-16T14:59:00.487 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-16T14:59:00.487 INFO:tasks.ceph:role: osd.2 2024-01-16T14:59:00.487 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi143.front.sepia.ceph.com 2024-01-16T14:59:00.487 DEBUG:teuthology.orchestra.run.smithi143:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-01-16T14:59:00.567 INFO:teuthology.orchestra.run.smithi143.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-01-16T14:59:00.567 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-16T14:59:00.567 INFO:teuthology.orchestra.run.smithi143.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-16T14:59:00.567 INFO:teuthology.orchestra.run.smithi143.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-16T14:59:00.567 INFO:teuthology.orchestra.run.smithi143.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-16T14:59:00.567 INFO:teuthology.orchestra.run.smithi143.stdout: = sunit=0 swidth=0 blks 2024-01-16T14:59:00.568 INFO:teuthology.orchestra.run.smithi143.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-16T14:59:00.568 INFO:teuthology.orchestra.run.smithi143.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-16T14:59:00.568 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-16T14:59:00.568 INFO:teuthology.orchestra.run.smithi143.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-16T14:59:01.080 INFO:teuthology.orchestra.run.smithi143.stdout:Discarding blocks...Done. 2024-01-16T14:59:01.086 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi143.front.sepia.ceph.com -o noatime 2024-01-16T14:59:01.086 DEBUG:teuthology.orchestra.run.smithi143:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-01-16T14:59:01.274 DEBUG:teuthology.orchestra.run.smithi143:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-01-16T14:59:01.380 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-01-16T14:59:01.470 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-16T14:59:01.470 INFO:tasks.ceph:role: osd.3 2024-01-16T14:59:01.470 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi143.front.sepia.ceph.com 2024-01-16T14:59:01.470 DEBUG:teuthology.orchestra.run.smithi143:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-01-16T14:59:01.564 INFO:teuthology.orchestra.run.smithi143.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-01-16T14:59:01.565 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-16T14:59:01.565 INFO:teuthology.orchestra.run.smithi143.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-16T14:59:01.565 INFO:teuthology.orchestra.run.smithi143.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-16T14:59:01.565 INFO:teuthology.orchestra.run.smithi143.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-16T14:59:01.565 INFO:teuthology.orchestra.run.smithi143.stdout: = sunit=0 swidth=0 blks 2024-01-16T14:59:01.565 INFO:teuthology.orchestra.run.smithi143.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-16T14:59:01.565 INFO:teuthology.orchestra.run.smithi143.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-16T14:59:01.565 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-16T14:59:01.565 INFO:teuthology.orchestra.run.smithi143.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-16T14:59:02.023 INFO:teuthology.orchestra.run.smithi143.stdout:Discarding blocks...Done. 2024-01-16T14:59:02.031 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi143.front.sepia.ceph.com -o noatime 2024-01-16T14:59:02.031 DEBUG:teuthology.orchestra.run.smithi143:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-01-16T14:59:02.178 DEBUG:teuthology.orchestra.run.smithi143:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-01-16T14:59:02.270 DEBUG:teuthology.orchestra.run.smithi143:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-16T14:59:02.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:02.427+0000 7fc6a3012500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-01-16T14:59:02.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:02.428+0000 7fc6a3012500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-01-16T14:59:02.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:02.428+0000 7fc6a3012500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-01-16T14:59:02.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:02.428+0000 7fc6a3012500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-01-16T14:59:02.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:02.428+0000 7fc6a3012500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-01-16T14:59:02.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:02.437+0000 7fc6a3012500 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-01-16T14:59:05.589 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-01-16T14:59:05.685 DEBUG:teuthology.orchestra.run.smithi143:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-16T14:59:05.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:05.828+0000 7fd6d4c22500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-01-16T14:59:05.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:05.830+0000 7fd6d4c22500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-01-16T14:59:05.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:05.830+0000 7fd6d4c22500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-01-16T14:59:05.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:05.830+0000 7fd6d4c22500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-01-16T14:59:05.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:05.830+0000 7fd6d4c22500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-01-16T14:59:05.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:05.832+0000 7fd6d4c22500 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-01-16T14:59:08.997 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-01-16T14:59:09.097 DEBUG:teuthology.orchestra.run.smithi143:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-16T14:59:09.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:09.252+0000 7f71ed81f500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-01-16T14:59:09.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:09.253+0000 7f71ed81f500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-01-16T14:59:09.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:09.253+0000 7f71ed81f500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-01-16T14:59:09.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:09.253+0000 7f71ed81f500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-01-16T14:59:09.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:09.253+0000 7f71ed81f500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-01-16T14:59:09.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:09.255+0000 7f71ed81f500 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-01-16T14:59:12.165 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-01-16T14:59:12.269 DEBUG:teuthology.orchestra.run.smithi143:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-16T14:59:12.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:12.424+0000 7fc19a420500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-01-16T14:59:12.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:12.425+0000 7fc19a420500 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-01-16T14:59:12.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:12.425+0000 7fc19a420500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-01-16T14:59:12.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:12.425+0000 7fc19a420500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-01-16T14:59:12.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:12.425+0000 7fc19a420500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-01-16T14:59:12.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:12.427+0000 7fc19a420500 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-01-16T14:59:15.575 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-01-16T14:59:15.668 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-01-16T14:59:15.766 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-16T14:59:15.767 INFO:tasks.ceph:role: osd.4 2024-01-16T14:59:15.767 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi204.front.sepia.ceph.com 2024-01-16T14:59:15.767 DEBUG:teuthology.orchestra.run.smithi204:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-01-16T14:59:15.865 INFO:teuthology.orchestra.run.smithi204.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-01-16T14:59:15.865 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-16T14:59:15.865 INFO:teuthology.orchestra.run.smithi204.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-16T14:59:15.865 INFO:teuthology.orchestra.run.smithi204.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-16T14:59:15.865 INFO:teuthology.orchestra.run.smithi204.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-16T14:59:15.865 INFO:teuthology.orchestra.run.smithi204.stdout: = sunit=0 swidth=0 blks 2024-01-16T14:59:15.865 INFO:teuthology.orchestra.run.smithi204.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-16T14:59:15.865 INFO:teuthology.orchestra.run.smithi204.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-16T14:59:15.865 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-16T14:59:15.865 INFO:teuthology.orchestra.run.smithi204.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-16T14:59:16.391 INFO:teuthology.orchestra.run.smithi204.stdout:Discarding blocks...Done. 2024-01-16T14:59:16.397 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi204.front.sepia.ceph.com -o noatime 2024-01-16T14:59:16.397 DEBUG:teuthology.orchestra.run.smithi204:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-01-16T14:59:16.584 DEBUG:teuthology.orchestra.run.smithi204:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-01-16T14:59:16.694 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-01-16T14:59:16.780 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-16T14:59:16.780 INFO:tasks.ceph:role: osd.5 2024-01-16T14:59:16.780 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi204.front.sepia.ceph.com 2024-01-16T14:59:16.780 DEBUG:teuthology.orchestra.run.smithi204:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-01-16T14:59:16.866 INFO:teuthology.orchestra.run.smithi204.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-01-16T14:59:16.866 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-16T14:59:16.867 INFO:teuthology.orchestra.run.smithi204.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-16T14:59:16.867 INFO:teuthology.orchestra.run.smithi204.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-16T14:59:16.867 INFO:teuthology.orchestra.run.smithi204.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-16T14:59:16.867 INFO:teuthology.orchestra.run.smithi204.stdout: = sunit=0 swidth=0 blks 2024-01-16T14:59:16.867 INFO:teuthology.orchestra.run.smithi204.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-16T14:59:16.867 INFO:teuthology.orchestra.run.smithi204.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-16T14:59:16.867 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-16T14:59:16.867 INFO:teuthology.orchestra.run.smithi204.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-16T14:59:17.345 INFO:teuthology.orchestra.run.smithi204.stdout:Discarding blocks...Done. 2024-01-16T14:59:17.350 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi204.front.sepia.ceph.com -o noatime 2024-01-16T14:59:17.350 DEBUG:teuthology.orchestra.run.smithi204:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-01-16T14:59:17.488 DEBUG:teuthology.orchestra.run.smithi204:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-01-16T14:59:17.574 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-01-16T14:59:17.651 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-16T14:59:17.652 INFO:tasks.ceph:role: osd.6 2024-01-16T14:59:17.652 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi204.front.sepia.ceph.com 2024-01-16T14:59:17.652 DEBUG:teuthology.orchestra.run.smithi204:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-01-16T14:59:17.730 INFO:teuthology.orchestra.run.smithi204.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-01-16T14:59:17.730 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-16T14:59:17.730 INFO:teuthology.orchestra.run.smithi204.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-16T14:59:17.731 INFO:teuthology.orchestra.run.smithi204.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-16T14:59:17.731 INFO:teuthology.orchestra.run.smithi204.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-16T14:59:17.731 INFO:teuthology.orchestra.run.smithi204.stdout: = sunit=0 swidth=0 blks 2024-01-16T14:59:17.731 INFO:teuthology.orchestra.run.smithi204.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-16T14:59:17.731 INFO:teuthology.orchestra.run.smithi204.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-16T14:59:17.731 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-16T14:59:17.731 INFO:teuthology.orchestra.run.smithi204.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-16T14:59:18.244 INFO:teuthology.orchestra.run.smithi204.stdout:Discarding blocks...Done. 2024-01-16T14:59:18.250 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi204.front.sepia.ceph.com -o noatime 2024-01-16T14:59:18.250 DEBUG:teuthology.orchestra.run.smithi204:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-01-16T14:59:18.432 DEBUG:teuthology.orchestra.run.smithi204:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-01-16T14:59:18.533 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-01-16T14:59:18.622 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-16T14:59:18.622 INFO:tasks.ceph:role: osd.7 2024-01-16T14:59:18.623 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi204.front.sepia.ceph.com 2024-01-16T14:59:18.623 DEBUG:teuthology.orchestra.run.smithi204:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-01-16T14:59:18.718 INFO:teuthology.orchestra.run.smithi204.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-01-16T14:59:18.719 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-16T14:59:18.719 INFO:teuthology.orchestra.run.smithi204.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-16T14:59:18.719 INFO:teuthology.orchestra.run.smithi204.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-16T14:59:18.719 INFO:teuthology.orchestra.run.smithi204.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-16T14:59:18.719 INFO:teuthology.orchestra.run.smithi204.stdout: = sunit=0 swidth=0 blks 2024-01-16T14:59:18.719 INFO:teuthology.orchestra.run.smithi204.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-16T14:59:18.719 INFO:teuthology.orchestra.run.smithi204.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-16T14:59:18.719 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-16T14:59:18.719 INFO:teuthology.orchestra.run.smithi204.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-16T14:59:19.184 INFO:teuthology.orchestra.run.smithi204.stdout:Discarding blocks...Done. 2024-01-16T14:59:19.190 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi204.front.sepia.ceph.com -o noatime 2024-01-16T14:59:19.190 DEBUG:teuthology.orchestra.run.smithi204:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-01-16T14:59:19.342 DEBUG:teuthology.orchestra.run.smithi204:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-01-16T14:59:19.434 DEBUG:teuthology.orchestra.run.smithi204:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-16T14:59:19.615 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:19.634+0000 7fbd2dc0c500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-01-16T14:59:19.620 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:19.639+0000 7fbd2dc0c500 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-01-16T14:59:19.620 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:19.639+0000 7fbd2dc0c500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-01-16T14:59:19.620 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:19.639+0000 7fbd2dc0c500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-01-16T14:59:19.620 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:19.639+0000 7fbd2dc0c500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-01-16T14:59:19.632 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:19.651+0000 7fbd2dc0c500 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-01-16T14:59:22.685 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T14:59:22.693 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T14:59:22.830 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-01-16T14:59:22.928 DEBUG:teuthology.orchestra.run.smithi204:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-16T14:59:23.069 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:23.088+0000 7faf9121e500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-01-16T14:59:23.070 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:23.089+0000 7faf9121e500 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-01-16T14:59:23.070 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:23.089+0000 7faf9121e500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-01-16T14:59:23.070 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:23.089+0000 7faf9121e500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-01-16T14:59:23.070 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:23.089+0000 7faf9121e500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-01-16T14:59:23.072 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:23.091+0000 7faf9121e500 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-01-16T14:59:25.991 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-01-16T14:59:26.086 DEBUG:teuthology.orchestra.run.smithi204:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-16T14:59:26.224 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:26.243+0000 7fa3bcad6500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-01-16T14:59:26.224 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:26.243+0000 7fa3bcad6500 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-01-16T14:59:26.224 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:26.243+0000 7fa3bcad6500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-01-16T14:59:26.224 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:26.243+0000 7fa3bcad6500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-01-16T14:59:26.224 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:26.243+0000 7fa3bcad6500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-01-16T14:59:26.226 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:26.246+0000 7fa3bcad6500 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-01-16T14:59:29.382 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-01-16T14:59:29.478 DEBUG:teuthology.orchestra.run.smithi204:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-16T14:59:29.623 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:29.642+0000 7fcbcb610500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-01-16T14:59:29.624 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:29.642+0000 7fcbcb610500 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-01-16T14:59:29.624 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:29.642+0000 7fcbcb610500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-01-16T14:59:29.624 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:29.642+0000 7fcbcb610500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-01-16T14:59:29.625 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:29.642+0000 7fcbcb610500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-01-16T14:59:29.625 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:29.644+0000 7fcbcb610500 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-01-16T14:59:32.794 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-01-16T14:59:32.882 INFO:tasks.ceph:Reading keys from all nodes... 2024-01-16T14:59:32.882 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T14:59:32.882 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-01-16T14:59:32.980 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T14:59:32.980 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-01-16T14:59:33.069 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T14:59:33.069 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-01-16T14:59:33.160 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T14:59:33.160 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-01-16T14:59:33.253 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T14:59:33.253 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-01-16T14:59:33.347 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T14:59:33.347 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-01-16T14:59:33.441 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T14:59:33.441 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-01-16T14:59:33.533 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-16T14:59:33.533 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-01-16T14:59:33.634 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-16T14:59:33.634 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-01-16T14:59:33.727 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-16T14:59:33.728 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-01-16T14:59:33.821 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-16T14:59:33.822 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-01-16T14:59:33.915 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-16T14:59:33.915 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-01-16T14:59:34.008 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-16T14:59:34.008 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-01-16T14:59:34.100 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-16T14:59:34.101 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-01-16T14:59:34.191 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T14:59:34.191 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-01-16T14:59:34.250 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T14:59:34.250 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-01-16T14:59:34.302 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-16T14:59:34.302 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-01-16T14:59:34.359 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-16T14:59:34.359 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-01-16T14:59:34.414 INFO:tasks.ceph:Adding keys to all mons... 2024-01-16T14:59:34.414 DEBUG:teuthology.orchestra.run.smithi143:> sudo tee -a /etc/ceph/ceph.keyring 2024-01-16T14:59:34.422 DEBUG:teuthology.orchestra.run.smithi204:> sudo tee -a /etc/ceph/ceph.keyring 2024-01-16T14:59:34.509 INFO:teuthology.orchestra.run.smithi204.stdout:[mgr.y] 2024-01-16T14:59:34.510 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCwmaZlwjghHBAA+NmdrcyEDELzm9asEMTSlw== 2024-01-16T14:59:34.510 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.a] 2024-01-16T14:59:34.510 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCwmaZlvfIyMxAAwY3gj2HmVRU8TVX6b7O0oQ== 2024-01-16T14:59:34.510 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.c] 2024-01-16T14:59:34.510 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCxmaZleyDIBxAApMtASK3s86++wYNAg2F2wQ== 2024-01-16T14:59:34.510 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.0] 2024-01-16T14:59:34.511 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQC2maZl9dWNGRAAvJgRYygFFMUKv+uEdeoQow== 2024-01-16T14:59:34.511 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.1] 2024-01-16T14:59:34.511 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQC5maZl0ER+MRAARG3PYOLij8sFyLEAFyKbAg== 2024-01-16T14:59:34.511 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.2] 2024-01-16T14:59:34.511 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQC9maZlUCoqDxAA9H2Sg4duizZJ/OtWGIvMoA== 2024-01-16T14:59:34.512 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.3] 2024-01-16T14:59:34.512 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQDAmaZlnilpGRAAONlgqwADjfzCMCWa7VL1Vw== 2024-01-16T14:59:34.512 INFO:teuthology.orchestra.run.smithi204.stdout:[mgr.x] 2024-01-16T14:59:34.512 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCwmaZleViaJxAAh27y3Kew/o1Lf+n25MN9gA== 2024-01-16T14:59:34.512 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.b] 2024-01-16T14:59:34.512 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCxmaZlDZ7gGBAAO4XA/MCFRinDyHETOLm6XA== 2024-01-16T14:59:34.513 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.d] 2024-01-16T14:59:34.513 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCxmaZllfS2KRAAsrbW5xlokeoer4d86zkoNg== 2024-01-16T14:59:34.513 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.4] 2024-01-16T14:59:34.513 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQDHmaZl51rfJRAA9rz9WdAcPwgIOYORvDUy/w== 2024-01-16T14:59:34.513 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.5] 2024-01-16T14:59:34.513 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQDLmaZlp5daBRAADKUPLUBspcDGWQJgIHhO/g== 2024-01-16T14:59:34.514 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.6] 2024-01-16T14:59:34.514 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQDOmaZlnBKMDhAAEgUx0WvuThbN7/OVL1symA== 2024-01-16T14:59:34.514 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.7] 2024-01-16T14:59:34.514 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQDRmaZlD5RUJhAARofymUD17OWQKVXv7LgSQw== 2024-01-16T14:59:34.514 INFO:teuthology.orchestra.run.smithi204.stdout:[client.0] 2024-01-16T14:59:34.514 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCxmaZlyhubNxAANh0FPLUzeCMMZcrjC6cocw== 2024-01-16T14:59:34.515 INFO:teuthology.orchestra.run.smithi204.stdout:[client.2] 2024-01-16T14:59:34.515 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCymaZlZnTGBhAAAHMSClKMYlOca2vYv2f5qw== 2024-01-16T14:59:34.515 INFO:teuthology.orchestra.run.smithi204.stdout:[client.1] 2024-01-16T14:59:34.515 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCymaZlO4qRERAA2wjMtfnNF/AHtt4vcfmSlg== 2024-01-16T14:59:34.515 INFO:teuthology.orchestra.run.smithi204.stdout:[client.3] 2024-01-16T14:59:34.515 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCymaZlJIomHBAASHAHcC+Ks0Nbf4BmJwTcWg== 2024-01-16T14:59:34.518 INFO:teuthology.orchestra.run.smithi143.stdout:[mgr.y] 2024-01-16T14:59:34.518 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQCwmaZlwjghHBAA+NmdrcyEDELzm9asEMTSlw== 2024-01-16T14:59:34.518 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.a] 2024-01-16T14:59:34.518 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQCwmaZlvfIyMxAAwY3gj2HmVRU8TVX6b7O0oQ== 2024-01-16T14:59:34.518 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.c] 2024-01-16T14:59:34.519 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQCxmaZleyDIBxAApMtASK3s86++wYNAg2F2wQ== 2024-01-16T14:59:34.519 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.0] 2024-01-16T14:59:34.519 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQC2maZl9dWNGRAAvJgRYygFFMUKv+uEdeoQow== 2024-01-16T14:59:34.519 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.1] 2024-01-16T14:59:34.519 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQC5maZl0ER+MRAARG3PYOLij8sFyLEAFyKbAg== 2024-01-16T14:59:34.519 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.2] 2024-01-16T14:59:34.519 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQC9maZlUCoqDxAA9H2Sg4duizZJ/OtWGIvMoA== 2024-01-16T14:59:34.519 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.3] 2024-01-16T14:59:34.520 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQDAmaZlnilpGRAAONlgqwADjfzCMCWa7VL1Vw== 2024-01-16T14:59:34.520 INFO:teuthology.orchestra.run.smithi143.stdout:[mgr.x] 2024-01-16T14:59:34.520 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQCwmaZleViaJxAAh27y3Kew/o1Lf+n25MN9gA== 2024-01-16T14:59:34.520 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.b] 2024-01-16T14:59:34.520 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQCxmaZlDZ7gGBAAO4XA/MCFRinDyHETOLm6XA== 2024-01-16T14:59:34.520 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.d] 2024-01-16T14:59:34.521 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQCxmaZllfS2KRAAsrbW5xlokeoer4d86zkoNg== 2024-01-16T14:59:34.521 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.4] 2024-01-16T14:59:34.521 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQDHmaZl51rfJRAA9rz9WdAcPwgIOYORvDUy/w== 2024-01-16T14:59:34.521 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.5] 2024-01-16T14:59:34.521 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQDLmaZlp5daBRAADKUPLUBspcDGWQJgIHhO/g== 2024-01-16T14:59:34.521 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.6] 2024-01-16T14:59:34.521 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQDOmaZlnBKMDhAAEgUx0WvuThbN7/OVL1symA== 2024-01-16T14:59:34.522 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.7] 2024-01-16T14:59:34.522 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQDRmaZlD5RUJhAARofymUD17OWQKVXv7LgSQw== 2024-01-16T14:59:34.522 INFO:teuthology.orchestra.run.smithi143.stdout:[client.0] 2024-01-16T14:59:34.522 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQCxmaZlyhubNxAANh0FPLUzeCMMZcrjC6cocw== 2024-01-16T14:59:34.522 INFO:teuthology.orchestra.run.smithi143.stdout:[client.2] 2024-01-16T14:59:34.522 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQCymaZlZnTGBhAAAHMSClKMYlOca2vYv2f5qw== 2024-01-16T14:59:34.523 INFO:teuthology.orchestra.run.smithi143.stdout:[client.1] 2024-01-16T14:59:34.523 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQCymaZlO4qRERAA2wjMtfnNF/AHtt4vcfmSlg== 2024-01-16T14:59:34.523 INFO:teuthology.orchestra.run.smithi143.stdout:[client.3] 2024-01-16T14:59:34.523 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQCymaZlJIomHBAASHAHcC+Ks0Nbf4BmJwTcWg== 2024-01-16T14:59:34.523 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-16T14:59:34.567 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-16T14:59:34.713 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-16T14:59:34.720 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-16T14:59:34.859 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-16T14:59:34.864 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-16T14:59:35.006 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-16T14:59:35.013 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-16T14:59:35.158 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-16T14:59:35.163 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-16T14:59:35.298 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-16T14:59:35.302 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-16T14:59:35.445 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-16T14:59:35.449 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-16T14:59:35.587 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-16T14:59:35.595 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-16T14:59:35.744 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-16T14:59:35.751 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-16T14:59:35.897 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-16T14:59:35.904 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-16T14:59:36.047 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-16T14:59:36.054 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-16T14:59:36.195 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-16T14:59:36.200 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-16T14:59:36.344 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-16T14:59:36.349 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-16T14:59:36.489 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-16T14:59:36.494 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-16T14:59:36.639 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-16T14:59:36.646 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-16T14:59:36.789 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-16T14:59:36.796 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-16T14:59:36.945 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-16T14:59:36.952 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-16T14:59:37.093 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-16T14:59:37.100 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-16T14:59:37.253 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-01-16T14:59:37.253 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-01-16T14:59:37.342 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-16T14:59:37.492 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-01-16T14:59:37.581 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-01-16T14:59:37.680 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-16T14:59:37.834 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-01-16T14:59:37.916 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-01-16T14:59:38.011 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-16T14:59:38.161 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-01-16T14:59:38.251 DEBUG:teuthology.orchestra.run.smithi143:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-01-16T14:59:38.259 DEBUG:teuthology.orchestra.run.smithi204:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-01-16T14:59:38.310 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-01-16T14:59:38.310 INFO:tasks.ceph.mon.a:Restarting daemon 2024-01-16T14:59:38.311 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-01-16T14:59:38.316 INFO:tasks.ceph.mon.a:Started 2024-01-16T14:59:38.316 INFO:tasks.ceph.mon.b:Restarting daemon 2024-01-16T14:59:38.316 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-01-16T14:59:38.321 INFO:tasks.ceph.mon.b:Started 2024-01-16T14:59:38.321 INFO:tasks.ceph.mon.c:Restarting daemon 2024-01-16T14:59:38.321 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-01-16T14:59:38.326 INFO:tasks.ceph.mon.c:Started 2024-01-16T14:59:38.326 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-01-16T14:59:38.326 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-01-16T14:59:38.326 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-01-16T14:59:38.331 INFO:tasks.ceph.mgr.y:Started 2024-01-16T14:59:38.331 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-01-16T14:59:38.331 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-01-16T14:59:38.335 INFO:tasks.ceph.mgr.x:Started 2024-01-16T14:59:38.336 INFO:tasks.ceph:Setting crush tunables to default 2024-01-16T14:59:38.336 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --cluster ceph osd crush tunables default 2024-01-16T14:59:38.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.527+0000 7f267da00640 1 Processor -- start 2024-01-16T14:59:38.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.527+0000 7f267da00640 1 -- start start 2024-01-16T14:59:38.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.528+0000 7f267da00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f267815b440 0x7f26780cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:38.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.528+0000 7f267da00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f26780c59b0 0x7f267806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:38.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.528+0000 7f267da00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26780c5dc0 con 0x7f26780cbf20 2024-01-16T14:59:38.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.528+0000 7f267da00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26780cc3d0 con 0x7f267815b440 2024-01-16T14:59:38.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.528+0000 7f267da00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f267815b820 con 0x7f26780c59b0 2024-01-16T14:59:38.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.529+0000 7f2676a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f267815b440 0x7f26780cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:38.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.529+0000 7f2676000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f26780cbf20 0x7f26780c52a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51652/0 (socket says 172.21.15.143:51652) 2024-01-16T14:59:38.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.529+0000 7f2676a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f267815b440 0x7f26780cb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52618/0 (socket says 172.21.15.143:52618) 2024-01-16T14:59:38.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.529+0000 7f2676000640 1 -- 172.21.15.143:0/229244612 learned_addr learned my addr 172.21.15.143:0/229244612 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T14:59:38.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.529+0000 7f2676a00640 1 -- 172.21.15.143:0/229244612 >> v2:172.21.15.143:3300/0 conn(0x7f267815b440 msgr2=0x7f26780cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-16T14:59:38.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.529+0000 7f2676a00640 1 -- 172.21.15.143:0/229244612 >> v2:172.21.15.143:3300/0 conn(0x7f267815b440 msgr2=0x7f26780cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-16T14:59:38.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.529+0000 7f2676a00640 1 --2- 172.21.15.143:0/229244612 >> v2:172.21.15.143:3300/0 conn(0x7f267815b440 0x7f26780cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-16T14:59:38.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.529+0000 7f2677400640 1 --2- 172.21.15.143:0/229244612 >> v2:172.21.15.204:3300/0 conn(0x7f26780c59b0 0x7f267806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:38.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.529+0000 7f2676a00640 1 --2- 172.21.15.143:0/229244612 >> v2:172.21.15.143:3300/0 conn(0x7f267815b440 0x7f26780cb9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-16T14:59:38.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.529+0000 7f2677400640 1 -- 172.21.15.143:0/229244612 >> v2:172.21.15.204:3300/0 conn(0x7f26780c59b0 msgr2=0x7f267806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-01-16T14:59:38.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.529+0000 7f2677400640 1 -- 172.21.15.143:0/229244612 >> v2:172.21.15.204:3300/0 conn(0x7f26780c59b0 msgr2=0x7f267806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-16T14:59:38.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.529+0000 7f2677400640 1 --2- 172.21.15.143:0/229244612 >> v2:172.21.15.204:3300/0 conn(0x7f26780c59b0 0x7f267806c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-16T14:59:38.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.529+0000 7f2677400640 1 --2- 172.21.15.143:0/229244612 >> v2:172.21.15.204:3300/0 conn(0x7f26780c59b0 0x7f267806c5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-16T14:59:38.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.544+0000 7f2675600640 1 -- 172.21.15.143:0/229244612 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1188415960 0 0) 0x7f26780c5dc0 con 0x7f26780cbf20 2024-01-16T14:59:38.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.544+0000 7f2675600640 1 -- 172.21.15.143:0/229244612 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2658003520 con 0x7f26780cbf20 2024-01-16T14:59:38.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.544+0000 7f2675600640 1 -- 172.21.15.143:0/229244612 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4205176644 0 0) 0x7f2658003520 con 0x7f26780cbf20 2024-01-16T14:59:38.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.544+0000 7f2675600640 1 -- 172.21.15.143:0/229244612 >> v2:172.21.15.204:3300/0 conn(0x7f26780c59b0 msgr2=0x7f267806c5f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-16T14:59:38.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.544+0000 7f2675600640 1 --2- 172.21.15.143:0/229244612 >> v2:172.21.15.204:3300/0 conn(0x7f26780c59b0 0x7f267806c5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:38.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.544+0000 7f2675600640 1 -- 172.21.15.143:0/229244612 >> v2:172.21.15.143:3300/0 conn(0x7f267815b440 msgr2=0x7f26780cb9e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-16T14:59:38.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.544+0000 7f2675600640 1 --2- 172.21.15.143:0/229244612 >> v2:172.21.15.143:3300/0 conn(0x7f267815b440 0x7f26780cb9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:38.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.544+0000 7f2675600640 1 -- 172.21.15.143:0/229244612 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f267815eea0 con 0x7f26780cbf20 2024-01-16T14:59:38.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.545+0000 7f2675600640 1 -- 172.21.15.143:0/229244612 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7f2660001670 con 0x7f26780cbf20 2024-01-16T14:59:38.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.545+0000 7f2675600640 1 -- 172.21.15.143:0/229244612 >> v1:172.21.15.143:6789/0 conn(0x7f26780cbf20 legacy=0x7f26780c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:38.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.545+0000 7f2675600640 1 --2- 172.21.15.143:0/229244612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2658003970 0x7f2658003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:38.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.545+0000 7f2675600640 1 --2- 172.21.15.143:0/229244612 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26580047c0 0x7f2658004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:38.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.545+0000 7f2675600640 1 --2- 172.21.15.143:0/229244612 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2658005180 0x7f265800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:38.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.545+0000 7f2675600640 1 -- 172.21.15.143:0/229244612 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26780cc3d0 con 0x7f2658005180 2024-01-16T14:59:38.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.545+0000 7f2675600640 1 -- 172.21.15.143:0/229244612 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267815b820 con 0x7f2658003970 2024-01-16T14:59:38.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.545+0000 7f2675600640 1 -- 172.21.15.143:0/229244612 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f265800c130 con 0x7f26580047c0 2024-01-16T14:59:38.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.545+0000 7f2676000640 1 --2- 172.21.15.143:0/229244612 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26580047c0 0x7f2658004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:38.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.545+0000 7f2676a00640 1 --2- 172.21.15.143:0/229244612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2658003970 0x7f2658003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:38.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.545+0000 7f2677400640 1 --2- 172.21.15.143:0/229244612 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2658005180 0x7f265800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:38.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.546+0000 7f2676000640 1 -- 172.21.15.143:0/229244612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2658003970 msgr2=0x7f2658003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:59:38.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.546+0000 7f2676000640 1 --2- 172.21.15.143:0/229244612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2658003970 0x7f2658003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:38.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.546+0000 7f2676000640 1 -- 172.21.15.143:0/229244612 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2658005180 msgr2=0x7f265800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:59:38.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.546+0000 7f2676000640 1 --2- 172.21.15.143:0/229244612 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2658005180 0x7f265800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:38.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.546+0000 7f2676000640 1 -- 172.21.15.143:0/229244612 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2658003520 con 0x7f26580047c0 2024-01-16T14:59:38.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.546+0000 7f2677400640 1 -- 172.21.15.143:0/229244612 reap_dead start 2024-01-16T14:59:38.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.546+0000 7f2676a00640 1 --2- 172.21.15.143:0/229244612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2658003970 0x7f2658003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:59:38.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.547+0000 7f2676000640 1 --2- 172.21.15.143:0/229244612 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26580047c0 0x7f2658004c40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f266000c290 tx=0x7f26580057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78b5b580e0693e5a server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:38.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.547+0000 7f2675600640 1 -- 172.21.15.143:0/229244612 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2660005ad0 con 0x7f26580047c0 2024-01-16T14:59:38.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.547+0000 7f2675600640 1 -- 172.21.15.143:0/229244612 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2660005c30 con 0x7f26580047c0 2024-01-16T14:59:38.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.547+0000 7f2675600640 1 -- 172.21.15.143:0/229244612 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266002c8d0 con 0x7f26580047c0 2024-01-16T14:59:38.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.548+0000 7f267da00640 1 -- 172.21.15.143:0/229244612 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26580047c0 msgr2=0x7f2658004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:38.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.548+0000 7f267da00640 1 --2- 172.21.15.143:0/229244612 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26580047c0 0x7f2658004c40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f266000c290 tx=0x7f26580057f0 comp rx=0 tx=0).stop 2024-01-16T14:59:38.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.548+0000 7f267da00640 1 -- 172.21.15.143:0/229244612 shutdown_connections 2024-01-16T14:59:38.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.548+0000 7f267da00640 1 --2- 172.21.15.143:0/229244612 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26580047c0 0x7f2658004c40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:38.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.548+0000 7f267da00640 1 -- 172.21.15.143:0/229244612 >> 172.21.15.143:0/229244612 conn(0x7f26780b0c50 msgr2=0x7f26780b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:59:38.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.548+0000 7f267da00640 1 -- 172.21.15.143:0/229244612 shutdown_connections 2024-01-16T14:59:38.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.548+0000 7f267da00640 1 -- 172.21.15.143:0/229244612 wait complete. 2024-01-16T14:59:38.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.550+0000 7f267da00640 1 Processor -- start 2024-01-16T14:59:38.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.550+0000 7f267da00640 1 -- start start 2024-01-16T14:59:38.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.551+0000 7f267da00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26580047c0 0x7f2678148430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:38.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.551+0000 7f267da00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f267814f8b0 0x7f26780afaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:38.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.551+0000 7f267da00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26780b0030 0x7f26781533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:38.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.551+0000 7f267da00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2678148970 con 0x7f26580047c0 2024-01-16T14:59:38.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.551+0000 7f267da00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26780b04e0 con 0x7f267814f8b0 2024-01-16T14:59:38.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.551+0000 7f267da00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f267814e810 con 0x7f26780b0030 2024-01-16T14:59:38.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.551+0000 7f2676a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26580047c0 0x7f2678148430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:38.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.551+0000 7f2676a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26580047c0 0x7f2678148430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52634/0 (socket says 172.21.15.143:52634) 2024-01-16T14:59:38.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.551+0000 7f2676a00640 1 -- 172.21.15.143:0/238988046 learned_addr learned my addr 172.21.15.143:0/238988046 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T14:59:38.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.551+0000 7f2677400640 1 --2- 172.21.15.143:0/238988046 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26780b0030 0x7f26781533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:38.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.551+0000 7f2676000640 1 --2- 172.21.15.143:0/238988046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f267814f8b0 0x7f26780afaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:38.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.551+0000 7f2676a00640 1 -- 172.21.15.143:0/238988046 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26780b0030 msgr2=0x7f26781533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:38.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.551+0000 7f2676a00640 1 --2- 172.21.15.143:0/238988046 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26780b0030 0x7f26781533e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:38.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.551+0000 7f2676a00640 1 -- 172.21.15.143:0/238988046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f267814f8b0 msgr2=0x7f26780afaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:38.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.551+0000 7f2676a00640 1 --2- 172.21.15.143:0/238988046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f267814f8b0 0x7f26780afaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:38.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.551+0000 7f2676a00640 1 -- 172.21.15.143:0/238988046 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f267814eaf0 con 0x7f26580047c0 2024-01-16T14:59:38.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.552+0000 7f2676a00640 1 --2- 172.21.15.143:0/238988046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26580047c0 0x7f2678148430 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f266800f4c0 tx=0x7f266800f980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:38.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.552+0000 7f266ea00640 1 -- 172.21.15.143:0/238988046 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2668014870 con 0x7f26580047c0 2024-01-16T14:59:38.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.552+0000 7f266ea00640 1 -- 172.21.15.143:0/238988046 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26680149d0 con 0x7f26580047c0 2024-01-16T14:59:38.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.552+0000 7f266ea00640 1 -- 172.21.15.143:0/238988046 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2668014da0 con 0x7f26580047c0 2024-01-16T14:59:38.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.552+0000 7f267da00640 1 -- 172.21.15.143:0/238988046 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f267814edd0 con 0x7f26580047c0 2024-01-16T14:59:38.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.552+0000 7f267da00640 1 -- 172.21.15.143:0/238988046 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f267814f390 con 0x7f26580047c0 2024-01-16T14:59:38.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.552+0000 7f266ea00640 1 -- 172.21.15.143:0/238988046 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f266801d700 con 0x7f26580047c0 2024-01-16T14:59:38.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.552+0000 7f266ea00640 1 -- 172.21.15.143:0/238988046 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2668015790 con 0x7f26580047c0 2024-01-16T14:59:38.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.553+0000 7f267da00640 1 -- 172.21.15.143:0/238988046 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f267804e8b0 con 0x7f26580047c0 2024-01-16T14:59:38.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.556+0000 7f266ea00640 1 -- 172.21.15.143:0/238988046 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f266803cce0 con 0x7f26580047c0 2024-01-16T14:59:38.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.627+0000 7f267da00640 1 -- 172.21.15.143:0/238988046 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f267806dc80 con 0x7f26580047c0 2024-01-16T14:59:38.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.633+0000 7f266ea00640 1 -- 172.21.15.143:0/238988046 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f266802b020 con 0x7f26580047c0 2024-01-16T14:59:38.614 INFO:teuthology.orchestra.run.smithi143.stderr:adjusted tunables profile to default 2024-01-16T14:59:38.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.634+0000 7f267da00640 1 -- 172.21.15.143:0/238988046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26580047c0 msgr2=0x7f2678148430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:38.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.634+0000 7f267da00640 1 --2- 172.21.15.143:0/238988046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26580047c0 0x7f2678148430 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f266800f4c0 tx=0x7f266800f980 comp rx=0 tx=0).stop 2024-01-16T14:59:38.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.634+0000 7f267da00640 1 -- 172.21.15.143:0/238988046 shutdown_connections 2024-01-16T14:59:38.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.634+0000 7f267da00640 1 --2- 172.21.15.143:0/238988046 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26780b0030 0x7f26781533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:38.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.634+0000 7f267da00640 1 --2- 172.21.15.143:0/238988046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f267814f8b0 0x7f26780afaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:38.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.634+0000 7f267da00640 1 --2- 172.21.15.143:0/238988046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26580047c0 0x7f2678148430 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:38.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.634+0000 7f267da00640 1 -- 172.21.15.143:0/238988046 >> 172.21.15.143:0/238988046 conn(0x7f26780b0c50 msgr2=0x7f26780c52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:59:38.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.634+0000 7f267da00640 1 -- 172.21.15.143:0/238988046 shutdown_connections 2024-01-16T14:59:38.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:38.634+0000 7f267da00640 1 -- 172.21.15.143:0/238988046 wait complete. 2024-01-16T14:59:38.633 INFO:tasks.ceph:check_enable_crimson: False 2024-01-16T14:59:38.634 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-01-16T14:59:38.634 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T14:59:38.634 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-01-16T14:59:38.714 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T14:59:38.714 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-01-16T14:59:38.832 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T14:59:38.832 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-01-16T14:59:38.956 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T14:59:38.956 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-01-16T14:59:39.085 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-16T14:59:39.086 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-01-16T14:59:39.180 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-16T14:59:39.180 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-01-16T14:59:39.264 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-16T14:59:39.264 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-01-16T14:59:39.285 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-16T14:59:39.305+0000 7feee7830040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-16T14:59:39.306 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T14:59:39.325+0000 7f27f9844040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-16T14:59:39.351 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-16T14:59:39.371+0000 7feee7830040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-16T14:59:39.364 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T14:59:39.383+0000 7f27f9844040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-16T14:59:39.376 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-16T14:59:39.376 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-01-16T14:59:39.504 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new f4c8891d-ab29-472a-907d-8446f20f7012 0 2024-01-16T14:59:39.690 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.709+0000 7fd899e00640 1 Processor -- start 2024-01-16T14:59:39.690 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.710+0000 7fd899e00640 1 -- start start 2024-01-16T14:59:39.692 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.711+0000 7fd899e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd8940b1ed0 0x7fd8941681d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:39.692 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.711+0000 7fd899e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd89415db00 con 0x7fd89415d720 2024-01-16T14:59:39.692 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.711+0000 7fd899e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd89406d1a0 con 0x7fd8940d3a40 2024-01-16T14:59:39.692 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.711+0000 7fd899e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8940d0ce0 con 0x7fd8940b1ed0 2024-01-16T14:59:39.692 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.711+0000 7fd88be00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fd8940d3a40 0x7fd89406ba80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:35898/0 (socket says 172.21.15.204:35898) 2024-01-16T14:59:39.692 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.711+0000 7fd893400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd8940b1ed0 0x7fd8941681d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:39.692 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.711+0000 7fd88be00640 1 -- 172.21.15.204:0/305200099 learned_addr learned my addr 172.21.15.204:0/305200099 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-01-16T14:59:39.692 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.711+0000 7fd892a00640 1 --1- 172.21.15.204:0/305200099 >> v1:172.21.15.143:6789/0 conn(0x7fd89415d720 0x7fd8940d3330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.204:57906/0 (socket says 172.21.15.204:57906) 2024-01-16T14:59:39.693 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.712+0000 7fd892000640 1 -- 172.21.15.204:0/305200099 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1411188462 0 0) 0x7fd89406d1a0 con 0x7fd8940d3a40 2024-01-16T14:59:39.693 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.712+0000 7fd892000640 1 -- 172.21.15.204:0/305200099 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd874003520 con 0x7fd8940d3a40 2024-01-16T14:59:39.693 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.712+0000 7fd893400640 1 -- 172.21.15.204:0/305200099 >> v1:172.21.15.204:6789/0 conn(0x7fd8940d3a40 legacy=0x7fd89406ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:39.693 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.712+0000 7fd893400640 1 -- 172.21.15.204:0/305200099 >> v1:172.21.15.143:6789/0 conn(0x7fd89415d720 legacy=0x7fd8940d3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:39.693 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.712+0000 7fd892000640 1 -- 172.21.15.204:0/305200099 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 368138064 0 0) 0x7fd89415db00 con 0x7fd89415d720 2024-01-16T14:59:39.693 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.712+0000 7fd893400640 1 -- 172.21.15.204:0/305200099 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd89406d400 con 0x7fd8940b1ed0 2024-01-16T14:59:39.693 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.713+0000 7fd893400640 1 --2- 172.21.15.204:0/305200099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8940b1ed0 0x7fd8941681d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd88c009d90 tx=0x7fd88c02beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c9a1e87422d9faa server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:39.694 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.713+0000 7fd892000640 1 -- 172.21.15.204:0/305200099 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88c02cc60 con 0x7fd8940b1ed0 2024-01-16T14:59:39.694 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.713+0000 7fd892000640 1 -- 172.21.15.204:0/305200099 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd88c02cdc0 con 0x7fd8940b1ed0 2024-01-16T14:59:39.694 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.713+0000 7fd892000640 1 -- 172.21.15.204:0/305200099 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88c034660 con 0x7fd8940b1ed0 2024-01-16T14:59:39.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.714+0000 7fd899e00640 1 -- 172.21.15.204:0/305200099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8940b1ed0 msgr2=0x7fd8941681d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:39.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.714+0000 7fd899e00640 1 --2- 172.21.15.204:0/305200099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8940b1ed0 0x7fd8941681d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd88c009d90 tx=0x7fd88c02beb0 comp rx=0 tx=0).stop 2024-01-16T14:59:39.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.714+0000 7fd899e00640 1 -- 172.21.15.204:0/305200099 shutdown_connections 2024-01-16T14:59:39.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.714+0000 7fd899e00640 1 --2- 172.21.15.204:0/305200099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8940b1ed0 0x7fd8941681d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:39.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.714+0000 7fd899e00640 1 -- 172.21.15.204:0/305200099 >> 172.21.15.204:0/305200099 conn(0x7fd89408af60 msgr2=0x7fd89409c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:59:39.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.714+0000 7fd899e00640 1 -- 172.21.15.204:0/305200099 shutdown_connections 2024-01-16T14:59:39.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.714+0000 7fd899e00640 1 -- 172.21.15.204:0/305200099 wait complete. 2024-01-16T14:59:39.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.716+0000 7fd899e00640 1 Processor -- start 2024-01-16T14:59:39.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.716+0000 7fd899e00640 1 -- start start 2024-01-16T14:59:39.698 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.717+0000 7fd899e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8940b1ed0 0x7fd8940c7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:39.698 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.717+0000 7fd899e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8940d3a40 0x7fd8940c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:39.698 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.717+0000 7fd899e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd89415d720 0x7fd894155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:39.698 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.717+0000 7fd899e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8940c9fb0 con 0x7fd8940d3a40 2024-01-16T14:59:39.699 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.717+0000 7fd899e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8940ca620 con 0x7fd89415d720 2024-01-16T14:59:39.699 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.717+0000 7fd899e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8940c7fd0 con 0x7fd8940b1ed0 2024-01-16T14:59:39.699 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.717+0000 7fd892a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8940b1ed0 0x7fd8940c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:39.699 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.717+0000 7fd892a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8940b1ed0 0x7fd8940c7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:47626/0 (socket says 172.21.15.204:47626) 2024-01-16T14:59:39.699 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.717+0000 7fd892a00640 1 -- 172.21.15.204:0/2970497567 learned_addr learned my addr 172.21.15.204:0/2970497567 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-01-16T14:59:39.699 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.717+0000 7fd893400640 1 --2- 172.21.15.204:0/2970497567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd89415d720 0x7fd894155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:39.699 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.718+0000 7fd88be00640 1 --2- 172.21.15.204:0/2970497567 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8940d3a40 0x7fd8940c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:39.699 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.718+0000 7fd892a00640 1 -- 172.21.15.204:0/2970497567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd89415d720 msgr2=0x7fd894155a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:39.699 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.718+0000 7fd892a00640 1 --2- 172.21.15.204:0/2970497567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd89415d720 0x7fd894155a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:39.699 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.718+0000 7fd892a00640 1 -- 172.21.15.204:0/2970497567 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8940d3a40 msgr2=0x7fd8940c9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:39.699 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.718+0000 7fd892a00640 1 --2- 172.21.15.204:0/2970497567 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8940d3a40 0x7fd8940c9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:39.699 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.718+0000 7fd892a00640 1 -- 172.21.15.204:0/2970497567 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd88c009a70 con 0x7fd8940b1ed0 2024-01-16T14:59:39.699 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.718+0000 7fd88be00640 1 --2- 172.21.15.204:0/2970497567 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8940d3a40 0x7fd8940c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:59:39.700 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.718+0000 7fd892a00640 1 --2- 172.21.15.204:0/2970497567 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8940b1ed0 0x7fd8940c7910 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd88400c500 tx=0x7fd88400c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:39.700 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.718+0000 7fd88b400640 1 -- 172.21.15.204:0/2970497567 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd884007560 con 0x7fd8940b1ed0 2024-01-16T14:59:39.700 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.718+0000 7fd88b400640 1 -- 172.21.15.204:0/2970497567 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8840047e0 con 0x7fd8940b1ed0 2024-01-16T14:59:39.701 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.718+0000 7fd88b400640 1 -- 172.21.15.204:0/2970497567 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd884004b70 con 0x7fd8940b1ed0 2024-01-16T14:59:39.701 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.718+0000 7fd899e00640 1 -- 172.21.15.204:0/2970497567 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8940c82b0 con 0x7fd8940b1ed0 2024-01-16T14:59:39.701 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.719+0000 7fd899e00640 1 -- 172.21.15.204:0/2970497567 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8940c8870 con 0x7fd8940b1ed0 2024-01-16T14:59:39.701 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.719+0000 7fd88b400640 1 -- 172.21.15.204:0/2970497567 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd884012400 con 0x7fd8940b1ed0 2024-01-16T14:59:39.701 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.719+0000 7fd88b400640 1 -- 172.21.15.204:0/2970497567 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd88401c480 con 0x7fd8940b1ed0 2024-01-16T14:59:39.701 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.720+0000 7fd899e00640 1 -- 172.21.15.204:0/2970497567 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd89404e8b0 con 0x7fd8940b1ed0 2024-01-16T14:59:39.705 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.724+0000 7fd88b400640 1 -- 172.21.15.204:0/2970497567 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fd884018070 con 0x7fd8940b1ed0 2024-01-16T14:59:39.783 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.802+0000 7fd899e00640 1 -- 172.21.15.204:0/2970497567 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f4c8891d-ab29-472a-907d-8446f20f7012", "id": 0} v 0) v1 -- 0x7fd8940b9030 con 0x7fd8940b1ed0 2024-01-16T14:59:39.790 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.809+0000 7fd88b400640 1 -- 172.21.15.204:0/2970497567 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f4c8891d-ab29-472a-907d-8446f20f7012", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd8840129c0 con 0x7fd8940b1ed0 2024-01-16T14:59:39.790 INFO:teuthology.orchestra.run.smithi204.stdout:0 2024-01-16T14:59:39.792 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.811+0000 7fd899e00640 1 -- 172.21.15.204:0/2970497567 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8940b1ed0 msgr2=0x7fd8940c7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:39.792 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.811+0000 7fd899e00640 1 --2- 172.21.15.204:0/2970497567 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8940b1ed0 0x7fd8940c7910 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd88400c500 tx=0x7fd88400c9c0 comp rx=0 tx=0).stop 2024-01-16T14:59:39.792 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.811+0000 7fd899e00640 1 -- 172.21.15.204:0/2970497567 shutdown_connections 2024-01-16T14:59:39.792 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.811+0000 7fd899e00640 1 --2- 172.21.15.204:0/2970497567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd89415d720 0x7fd894155a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:39.792 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.811+0000 7fd899e00640 1 --2- 172.21.15.204:0/2970497567 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8940d3a40 0x7fd8940c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:39.792 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.811+0000 7fd899e00640 1 --2- 172.21.15.204:0/2970497567 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8940b1ed0 0x7fd8940c7910 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:39.792 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.811+0000 7fd899e00640 1 -- 172.21.15.204:0/2970497567 >> 172.21.15.204:0/2970497567 conn(0x7fd89408af60 msgr2=0x7fd89415cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:59:39.792 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.811+0000 7fd899e00640 1 -- 172.21.15.204:0/2970497567 shutdown_connections 2024-01-16T14:59:39.792 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:39.811+0000 7fd899e00640 1 -- 172.21.15.204:0/2970497567 wait complete. 2024-01-16T14:59:39.808 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new c4a2788d-8b89-4992-82b2-7f4b9debd509 1 2024-01-16T14:59:39.977 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T14:59:39.996+0000 7f27f9844040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-16T14:59:39.991 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-16T14:59:40.010+0000 7feee7830040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-16T14:59:39.996 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.016+0000 7fa2b7000640 1 Processor -- start 2024-01-16T14:59:39.996 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.016+0000 7fa2b7000640 1 -- start start 2024-01-16T14:59:39.998 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.017+0000 7fa2b7000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa2b0160a00 0x7fa2b00d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:39.998 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.017+0000 7fa2b7000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2b00d1810 con 0x7fa2b00d36d0 2024-01-16T14:59:39.998 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.017+0000 7fa2b7000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2b00d1ae0 con 0x7fa2b00d1380 2024-01-16T14:59:39.998 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.018+0000 7fa2b7000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2b00d1dc0 con 0x7fa2b0160a00 2024-01-16T14:59:39.998 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.018+0000 7fa2b5c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa2b0160a00 0x7fa2b00d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:39.999 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.018+0000 7fa2b5200640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fa2b00d1380 0x7fa2b00d2fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:35914/0 (socket says 172.21.15.204:35914) 2024-01-16T14:59:39.999 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.018+0000 7fa2b5200640 1 -- 172.21.15.204:0/218364966 learned_addr learned my addr 172.21.15.204:0/218364966 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-01-16T14:59:39.999 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.018+0000 7fa2b5c00640 1 -- 172.21.15.204:0/218364966 >> v1:172.21.15.204:6789/0 conn(0x7fa2b00d1380 legacy=0x7fa2b00d2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:39.999 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.018+0000 7fa2b5c00640 1 -- 172.21.15.204:0/218364966 >> v1:172.21.15.143:6789/0 conn(0x7fa2b00d36d0 legacy=0x7fa2b00ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:39.999 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.018+0000 7fa2afe00640 1 -- 172.21.15.204:0/218364966 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 168942583 0 0) 0x7fa2b00d1ae0 con 0x7fa2b00d1380 2024-01-16T14:59:39.999 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.018+0000 7fa2b5c00640 1 -- 172.21.15.204:0/218364966 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b00d20a0 con 0x7fa2b0160a00 2024-01-16T14:59:40.000 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.019+0000 7fa2b5c00640 1 --2- 172.21.15.204:0/218364966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2b0160a00 0x7fa2b00d0e40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00097a0 tx=0x7fa2a002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=753f5a324f79aa9c server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:40.000 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.020+0000 7fa2afe00640 1 -- 172.21.15.204:0/218364966 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a002e070 con 0x7fa2b0160a00 2024-01-16T14:59:40.000 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.020+0000 7fa2afe00640 1 -- 172.21.15.204:0/218364966 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a00315a0 con 0x7fa2b0160a00 2024-01-16T14:59:40.000 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.020+0000 7fa2afe00640 1 -- 172.21.15.204:0/218364966 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0031930 con 0x7fa2b0160a00 2024-01-16T14:59:40.001 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.020+0000 7fa2b7000640 1 -- 172.21.15.204:0/218364966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2b0160a00 msgr2=0x7fa2b00d0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:40.001 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.020+0000 7fa2b7000640 1 --2- 172.21.15.204:0/218364966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2b0160a00 0x7fa2b00d0e40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00097a0 tx=0x7fa2a002b840 comp rx=0 tx=0).stop 2024-01-16T14:59:40.001 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.020+0000 7fa2b7000640 1 -- 172.21.15.204:0/218364966 shutdown_connections 2024-01-16T14:59:40.001 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.020+0000 7fa2b7000640 1 --2- 172.21.15.204:0/218364966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2b0160a00 0x7fa2b00d0e40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:40.001 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.020+0000 7fa2b7000640 1 -- 172.21.15.204:0/218364966 >> 172.21.15.204:0/218364966 conn(0x7fa2b008cc90 msgr2=0x7fa2b009c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:59:40.002 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.021+0000 7fa2b7000640 1 -- 172.21.15.204:0/218364966 shutdown_connections 2024-01-16T14:59:40.002 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.021+0000 7fa2b7000640 1 -- 172.21.15.204:0/218364966 wait complete. 2024-01-16T14:59:40.004 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.023+0000 7fa2b7000640 1 Processor -- start 2024-01-16T14:59:40.004 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.023+0000 7fa2b7000640 1 -- start start 2024-01-16T14:59:40.007 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.025+0000 7fa2b7000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa2b00d1380 0x7fa2b00a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:40.007 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.025+0000 7fa2b7000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2b00d36d0 0x7fa2b01f6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:40.007 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.025+0000 7fa2b7000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2b0160a00 0x7fa2b01f69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:40.007 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.025+0000 7fa2b7000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b01f7380 con 0x7fa2b00d36d0 2024-01-16T14:59:40.007 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.025+0000 7fa2b7000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b01f4160 con 0x7fa2b0160a00 2024-01-16T14:59:40.007 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.025+0000 7fa2b7000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b00c8400 con 0x7fa2b00d1380 2024-01-16T14:59:40.007 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.025+0000 7fa2b5c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa2b00d1380 0x7fa2b00a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:40.007 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.025+0000 7fa2b5c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa2b00d1380 0x7fa2b00a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:47642/0 (socket says 172.21.15.204:47642) 2024-01-16T14:59:40.007 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.025+0000 7fa2b5c00640 1 -- 172.21.15.204:0/2629982285 learned_addr learned my addr 172.21.15.204:0/2629982285 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-01-16T14:59:40.007 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.025+0000 7fa2b6600640 1 --2- 172.21.15.204:0/2629982285 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2b0160a00 0x7fa2b01f69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:40.007 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.025+0000 7fa2b5200640 1 --2- 172.21.15.204:0/2629982285 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2b00d36d0 0x7fa2b01f6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:40.008 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.025+0000 7fa2b5c00640 1 -- 172.21.15.204:0/2629982285 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2b0160a00 msgr2=0x7fa2b01f69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:40.008 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.025+0000 7fa2b5c00640 1 --2- 172.21.15.204:0/2629982285 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2b0160a00 0x7fa2b01f69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:40.008 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.025+0000 7fa2b5c00640 1 -- 172.21.15.204:0/2629982285 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2b00d36d0 msgr2=0x7fa2b01f6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:40.008 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.026+0000 7fa2b5c00640 1 --2- 172.21.15.204:0/2629982285 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2b00d36d0 0x7fa2b01f6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:40.008 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.026+0000 7fa2b5c00640 1 -- 172.21.15.204:0/2629982285 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a0009400 con 0x7fa2b00d1380 2024-01-16T14:59:40.008 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.026+0000 7fa2b5c00640 1 --2- 172.21.15.204:0/2629982285 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa2b00d1380 0x7fa2b00a0740 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0034040 tx=0x7fa2a00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:40.008 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.026+0000 7fa2ad600640 1 -- 172.21.15.204:0/2629982285 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0032900 con 0x7fa2b00d1380 2024-01-16T14:59:40.008 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.026+0000 7fa2ad600640 1 -- 172.21.15.204:0/2629982285 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a0032a80 con 0x7fa2b00d1380 2024-01-16T14:59:40.008 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.026+0000 7fa2ad600640 1 -- 172.21.15.204:0/2629982285 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0032e30 con 0x7fa2b00d1380 2024-01-16T14:59:40.008 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.026+0000 7fa2b7000640 1 -- 172.21.15.204:0/2629982285 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b00c86e0 con 0x7fa2b00d1380 2024-01-16T14:59:40.008 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.026+0000 7fa2b7000640 1 -- 172.21.15.204:0/2629982285 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b00c8ca0 con 0x7fa2b00d1380 2024-01-16T14:59:40.008 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.027+0000 7fa2ad600640 1 -- 172.21.15.204:0/2629982285 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa2a0005c90 con 0x7fa2b00d1380 2024-01-16T14:59:40.009 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.027+0000 7fa2ad600640 1 -- 172.21.15.204:0/2629982285 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fa2a00457a0 con 0x7fa2b00d1380 2024-01-16T14:59:40.009 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.027+0000 7fa2b7000640 1 -- 172.21.15.204:0/2629982285 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2b004e840 con 0x7fa2b00d1380 2024-01-16T14:59:40.010 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.030+0000 7fa2ad600640 1 -- 172.21.15.204:0/2629982285 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fa2a0063ce0 con 0x7fa2b00d1380 2024-01-16T14:59:40.038 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T14:59:40.057+0000 7f27f9844040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-16T14:59:40.058 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-16T14:59:40.077+0000 7feee7830040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-16T14:59:40.088 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.108+0000 7fa2b7000640 1 -- 172.21.15.204:0/2629982285 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c4a2788d-8b89-4992-82b2-7f4b9debd509", "id": 1} v 0) v1 -- 0x7fa2b015f5a0 con 0x7fa2b00d1380 2024-01-16T14:59:40.089 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T14:59:40.108+0000 7f27f9844040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-16T14:59:40.096 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.116+0000 7fa2ad600640 1 -- 172.21.15.204:0/2629982285 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c4a2788d-8b89-4992-82b2-7f4b9debd509", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa2a0052070 con 0x7fa2b00d1380 2024-01-16T14:59:40.096 INFO:teuthology.orchestra.run.smithi204.stdout:1 2024-01-16T14:59:40.098 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.117+0000 7fa2b7000640 1 -- 172.21.15.204:0/2629982285 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa2b00d1380 msgr2=0x7fa2b00a0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:40.098 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.117+0000 7fa2b7000640 1 --2- 172.21.15.204:0/2629982285 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa2b00d1380 0x7fa2b00a0740 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0034040 tx=0x7fa2a00046c0 comp rx=0 tx=0).stop 2024-01-16T14:59:40.098 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.117+0000 7fa2b7000640 1 -- 172.21.15.204:0/2629982285 shutdown_connections 2024-01-16T14:59:40.098 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.117+0000 7fa2b7000640 1 --2- 172.21.15.204:0/2629982285 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2b0160a00 0x7fa2b01f69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:40.098 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.117+0000 7fa2b7000640 1 --2- 172.21.15.204:0/2629982285 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2b00d36d0 0x7fa2b01f6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:40.098 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.117+0000 7fa2b7000640 1 --2- 172.21.15.204:0/2629982285 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa2b00d1380 0x7fa2b00a0740 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:40.098 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.117+0000 7fa2b7000640 1 -- 172.21.15.204:0/2629982285 >> 172.21.15.204:0/2629982285 conn(0x7fa2b008cc90 msgr2=0x7fa2b006bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:59:40.098 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.117+0000 7fa2b7000640 1 -- 172.21.15.204:0/2629982285 shutdown_connections 2024-01-16T14:59:40.098 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.117+0000 7fa2b7000640 1 -- 172.21.15.204:0/2629982285 wait complete. 2024-01-16T14:59:40.112 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-16T14:59:40.131+0000 7feee7830040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-16T14:59:40.114 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 64645c1d-31b2-4794-b0f2-7ccf8df30393 2 2024-01-16T14:59:40.151 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T14:59:40.170+0000 7f27f9844040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-16T14:59:40.177 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-16T14:59:40.197+0000 7feee7830040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-16T14:59:40.200 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T14:59:40.218+0000 7f27f9844040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-16T14:59:40.230 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-16T14:59:40.249+0000 7feee7830040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-16T14:59:40.297 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.316+0000 7fdc77e00640 1 Processor -- start 2024-01-16T14:59:40.297 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.316+0000 7fdc77e00640 1 -- start start 2024-01-16T14:59:40.299 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.318+0000 7fdc77e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdc78160a00 0x7fdc780d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:40.299 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.318+0000 7fdc77e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc780d1380 0x7fdc780d2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:40.299 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.318+0000 7fdc77e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc780d1ae0 con 0x7fdc780d3500 2024-01-16T14:59:40.299 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.318+0000 7fdc77e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc780d1dc0 con 0x7fdc78160a00 2024-01-16T14:59:40.300 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.318+0000 7fdc77e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc780d20a0 con 0x7fdc780d1380 2024-01-16T14:59:40.300 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.319+0000 7fdc6fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc780d1380 0x7fdc780d2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:40.300 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.319+0000 7fdc76a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdc78160a00 0x7fdc780d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:40.300 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.319+0000 7fdc77400640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fdc780d3500 0x7fdc780cfc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:35926/0 (socket says 172.21.15.204:35926) 2024-01-16T14:59:40.300 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.319+0000 7fdc77400640 1 -- 172.21.15.204:0/623681292 learned_addr learned my addr 172.21.15.204:0/623681292 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-01-16T14:59:40.300 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.319+0000 7fdc6fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc780d1380 0x7fdc780d2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:47654/0 (socket says 172.21.15.204:47654) 2024-01-16T14:59:40.301 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.319+0000 7fdc76a00640 1 -- 172.21.15.204:0/623681292 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc780d1380 msgr2=0x7fdc780d2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:59:40.301 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.319+0000 7fdc76a00640 1 --2- 172.21.15.204:0/623681292 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc780d1380 0x7fdc780d2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:40.301 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.319+0000 7fdc76a00640 1 -- 172.21.15.204:0/623681292 >> v1:172.21.15.204:6789/0 conn(0x7fdc780d3500 legacy=0x7fdc780cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:40.301 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.319+0000 7fdc76a00640 1 -- 172.21.15.204:0/623681292 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc780ce140 con 0x7fdc78160a00 2024-01-16T14:59:40.301 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.319+0000 7fdc6fe00640 1 --2- 172.21.15.204:0/623681292 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc780d1380 0x7fdc780d2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:59:40.301 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.320+0000 7fdc76a00640 1 --2- 172.21.15.204:0/623681292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc78160a00 0x7fdc780d0e40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdc680097a0 tx=0x7fdc6802b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad3d3523e5e5874c server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:40.302 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.321+0000 7fdc76000640 1 -- 172.21.15.204:0/623681292 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc680316f0 con 0x7fdc78160a00 2024-01-16T14:59:40.302 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.321+0000 7fdc76000640 1 -- 172.21.15.204:0/623681292 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc68031850 con 0x7fdc78160a00 2024-01-16T14:59:40.302 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.321+0000 7fdc76000640 1 -- 172.21.15.204:0/623681292 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc68031be0 con 0x7fdc78160a00 2024-01-16T14:59:40.303 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.321+0000 7fdc77e00640 1 -- 172.21.15.204:0/623681292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc78160a00 msgr2=0x7fdc780d0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:40.303 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.321+0000 7fdc77e00640 1 --2- 172.21.15.204:0/623681292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc78160a00 0x7fdc780d0e40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdc680097a0 tx=0x7fdc6802b790 comp rx=0 tx=0).stop 2024-01-16T14:59:40.303 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.321+0000 7fdc77e00640 1 -- 172.21.15.204:0/623681292 shutdown_connections 2024-01-16T14:59:40.303 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.321+0000 7fdc77e00640 1 --2- 172.21.15.204:0/623681292 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc780d1380 0x7fdc780d2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:40.303 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.321+0000 7fdc77e00640 1 --2- 172.21.15.204:0/623681292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc78160a00 0x7fdc780d0e40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:40.303 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.321+0000 7fdc77e00640 1 -- 172.21.15.204:0/623681292 >> 172.21.15.204:0/623681292 conn(0x7fdc7808cc90 msgr2=0x7fdc7809c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:59:40.303 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.322+0000 7fdc77e00640 1 -- 172.21.15.204:0/623681292 shutdown_connections 2024-01-16T14:59:40.303 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.322+0000 7fdc77e00640 1 -- 172.21.15.204:0/623681292 wait complete. 2024-01-16T14:59:40.305 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.324+0000 7fdc77e00640 1 Processor -- start 2024-01-16T14:59:40.305 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.324+0000 7fdc77e00640 1 -- start start 2024-01-16T14:59:40.307 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.326+0000 7fdc77e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc780d1380 0x7fdc780b7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:40.307 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.326+0000 7fdc77e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc780d3500 0x7fdc78149240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:40.307 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.326+0000 7fdc77e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc78160a00 0x7fdc78149780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:40.307 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.326+0000 7fdc77e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc780b8cc0 con 0x7fdc780d1380 2024-01-16T14:59:40.307 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.326+0000 7fdc77e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc780b86b0 con 0x7fdc780d3500 2024-01-16T14:59:40.307 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.326+0000 7fdc77e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc780b6280 con 0x7fdc78160a00 2024-01-16T14:59:40.309 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.326+0000 7fdc6fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc780d3500 0x7fdc78149240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:40.309 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.326+0000 7fdc77400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc78160a00 0x7fdc78149780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:40.309 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.326+0000 7fdc6fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc780d3500 0x7fdc78149240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:44850/0 (socket says 172.21.15.204:44850) 2024-01-16T14:59:40.309 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.326+0000 7fdc77400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc78160a00 0x7fdc78149780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:47662/0 (socket says 172.21.15.204:47662) 2024-01-16T14:59:40.309 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.326+0000 7fdc6fe00640 1 -- 172.21.15.204:0/2313836233 learned_addr learned my addr 172.21.15.204:0/2313836233 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-01-16T14:59:40.309 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.326+0000 7fdc76a00640 1 --2- 172.21.15.204:0/2313836233 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc780d1380 0x7fdc780b7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:40.309 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.327+0000 7fdc77400640 1 -- 172.21.15.204:0/2313836233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc780d3500 msgr2=0x7fdc78149240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:40.309 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.327+0000 7fdc77400640 1 --2- 172.21.15.204:0/2313836233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc780d3500 0x7fdc78149240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:40.309 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.327+0000 7fdc77400640 1 -- 172.21.15.204:0/2313836233 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc780d1380 msgr2=0x7fdc780b7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:40.309 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.327+0000 7fdc77400640 1 --2- 172.21.15.204:0/2313836233 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc780d1380 0x7fdc780b7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:40.310 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.327+0000 7fdc77400640 1 -- 172.21.15.204:0/2313836233 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc60009400 con 0x7fdc78160a00 2024-01-16T14:59:40.310 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.327+0000 7fdc76a00640 1 --2- 172.21.15.204:0/2313836233 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc780d1380 0x7fdc780b7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:59:40.310 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.327+0000 7fdc77400640 1 --2- 172.21.15.204:0/2313836233 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc78160a00 0x7fdc78149780 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdc7000caa0 tx=0x7fdc7000cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:40.310 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.327+0000 7fdc6ea00640 1 -- 172.21.15.204:0/2313836233 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7000ab40 con 0x7fdc78160a00 2024-01-16T14:59:40.310 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.327+0000 7fdc6ea00640 1 -- 172.21.15.204:0/2313836233 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc7000aca0 con 0x7fdc78160a00 2024-01-16T14:59:40.310 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.327+0000 7fdc6ea00640 1 -- 172.21.15.204:0/2313836233 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc70012600 con 0x7fdc78160a00 2024-01-16T14:59:40.310 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.327+0000 7fdc77e00640 1 -- 172.21.15.204:0/2313836233 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc68009400 con 0x7fdc78160a00 2024-01-16T14:59:40.310 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.327+0000 7fdc77e00640 1 -- 172.21.15.204:0/2313836233 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc780b6560 con 0x7fdc78160a00 2024-01-16T14:59:40.310 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.328+0000 7fdc6ea00640 1 -- 172.21.15.204:0/2313836233 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fdc70012a60 con 0x7fdc78160a00 2024-01-16T14:59:40.310 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.328+0000 7fdc6ea00640 1 -- 172.21.15.204:0/2313836233 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fdc7001ce00 con 0x7fdc78160a00 2024-01-16T14:59:40.310 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.328+0000 7fdc77e00640 1 -- 172.21.15.204:0/2313836233 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc7804e840 con 0x7fdc78160a00 2024-01-16T14:59:40.312 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T14:59:40.331+0000 7f27f9844040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-16T14:59:40.313 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.333+0000 7fdc6ea00640 1 -- 172.21.15.204:0/2313836233 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fdc70021060 con 0x7fdc78160a00 2024-01-16T14:59:40.355 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-16T14:59:40.375+0000 7feee7830040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-16T14:59:40.391 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.410+0000 7fdc77e00640 1 -- 172.21.15.204:0/2313836233 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd new", "uuid": "64645c1d-31b2-4794-b0f2-7ccf8df30393", "id": 2} v 0) v1 -- 0x7fdc780b0c20 con 0x7fdc78160a00 2024-01-16T14:59:40.391 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T14:59:40.409+0000 7f27f9844040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-16T14:59:40.399 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.418+0000 7fdc6ea00640 1 -- 172.21.15.204:0/2313836233 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "64645c1d-31b2-4794-b0f2-7ccf8df30393", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fdc70017070 con 0x7fdc78160a00 2024-01-16T14:59:40.399 INFO:teuthology.orchestra.run.smithi204.stdout:2 2024-01-16T14:59:40.400 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.419+0000 7fdc77e00640 1 -- 172.21.15.204:0/2313836233 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc78160a00 msgr2=0x7fdc78149780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:40.400 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.419+0000 7fdc77e00640 1 --2- 172.21.15.204:0/2313836233 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc78160a00 0x7fdc78149780 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdc7000caa0 tx=0x7fdc7000cf60 comp rx=0 tx=0).stop 2024-01-16T14:59:40.400 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.419+0000 7fdc77e00640 1 -- 172.21.15.204:0/2313836233 shutdown_connections 2024-01-16T14:59:40.400 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.419+0000 7fdc77e00640 1 --2- 172.21.15.204:0/2313836233 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc78160a00 0x7fdc78149780 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:40.401 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.419+0000 7fdc77e00640 1 --2- 172.21.15.204:0/2313836233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc780d3500 0x7fdc78149240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:40.401 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.419+0000 7fdc77e00640 1 --2- 172.21.15.204:0/2313836233 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc780d1380 0x7fdc780b7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:40.401 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.419+0000 7fdc77e00640 1 -- 172.21.15.204:0/2313836233 >> 172.21.15.204:0/2313836233 conn(0x7fdc7808cc90 msgr2=0x7fdc780d0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:59:40.401 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.420+0000 7fdc77e00640 1 -- 172.21.15.204:0/2313836233 shutdown_connections 2024-01-16T14:59:40.401 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.420+0000 7fdc77e00640 1 -- 172.21.15.204:0/2313836233 wait complete. 2024-01-16T14:59:40.415 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 73674129-a84e-4d31-8247-7d0179ae6df2 3 2024-01-16T14:59:40.441 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-16T14:59:40.460+0000 7feee7830040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-16T14:59:40.444 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T14:59:40.463+0000 7f27f9844040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-16T14:59:40.498 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-16T14:59:40.517+0000 7feee7830040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-16T14:59:40.503 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T14:59:40.522+0000 7f27f9844040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-16T14:59:40.559 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T14:59:40.577+0000 7f27f9844040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-16T14:59:40.563 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-16T14:59:40.582+0000 7feee7830040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-16T14:59:40.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.622+0000 7fee7dc00640 1 Processor -- start 2024-01-16T14:59:40.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.622+0000 7fee7dc00640 1 -- start start 2024-01-16T14:59:40.605 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.623+0000 7fee7dc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee780cb850 0x7fee780cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:40.605 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.623+0000 7fee7dc00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fee780b40d0 0x7fee780cf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:40.605 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.623+0000 7fee7dc00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee780cc170 con 0x7fee780d0a00 2024-01-16T14:59:40.605 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.623+0000 7fee7dc00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee78070680 con 0x7fee780b40d0 2024-01-16T14:59:40.605 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.623+0000 7fee7dc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee78070870 con 0x7fee780cb850 2024-01-16T14:59:40.605 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.623+0000 7fee76a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee780cb850 0x7fee780cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:40.605 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.623+0000 7fee6fe00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fee780d0a00 0x7fee7806ff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:35936/0 (socket says 172.21.15.204:35936) 2024-01-16T14:59:40.605 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.623+0000 7fee6fe00640 1 -- 172.21.15.204:0/1323993745 learned_addr learned my addr 172.21.15.204:0/1323993745 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-01-16T14:59:40.605 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.624+0000 7fee77400640 1 --2- 172.21.15.204:0/1323993745 >> v2:172.21.15.143:3300/0 conn(0x7fee780b40d0 0x7fee780cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:40.605 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.624+0000 7fee76a00640 1 -- 172.21.15.204:0/1323993745 >> v2:172.21.15.143:3300/0 conn(0x7fee780b40d0 msgr2=0x7fee780cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:59:40.605 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.624+0000 7fee76a00640 1 --2- 172.21.15.204:0/1323993745 >> v2:172.21.15.143:3300/0 conn(0x7fee780b40d0 0x7fee780cf670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:40.605 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.624+0000 7fee76a00640 1 -- 172.21.15.204:0/1323993745 >> v1:172.21.15.204:6789/0 conn(0x7fee780d0a00 legacy=0x7fee7806ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:40.605 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.624+0000 7fee76a00640 1 -- 172.21.15.204:0/1323993745 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee78070b50 con 0x7fee780cb850 2024-01-16T14:59:40.605 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.624+0000 7fee76000640 1 -- 172.21.15.204:0/1323993745 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1919832485 0 0) 0x7fee780cc170 con 0x7fee780d0a00 2024-01-16T14:59:40.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.626+0000 7fee76a00640 1 --2- 172.21.15.204:0/1323993745 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee780cb850 0x7fee780cbc30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fee680097a0 tx=0x7fee6802b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c33c38f84092b870 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:40.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.626+0000 7fee76000640 1 -- 172.21.15.204:0/1323993745 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6802e070 con 0x7fee780cb850 2024-01-16T14:59:40.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.626+0000 7fee76000640 1 -- 172.21.15.204:0/1323993745 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee680315a0 con 0x7fee780cb850 2024-01-16T14:59:40.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.626+0000 7fee76000640 1 -- 172.21.15.204:0/1323993745 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee68031930 con 0x7fee780cb850 2024-01-16T14:59:40.608 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.627+0000 7fee7dc00640 1 -- 172.21.15.204:0/1323993745 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee780cb850 msgr2=0x7fee780cbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:40.608 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.627+0000 7fee7dc00640 1 --2- 172.21.15.204:0/1323993745 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee780cb850 0x7fee780cbc30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fee680097a0 tx=0x7fee6802b840 comp rx=0 tx=0).stop 2024-01-16T14:59:40.608 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.627+0000 7fee7dc00640 1 -- 172.21.15.204:0/1323993745 shutdown_connections 2024-01-16T14:59:40.608 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.627+0000 7fee7dc00640 1 --2- 172.21.15.204:0/1323993745 >> v2:172.21.15.143:3300/0 conn(0x7fee780b40d0 0x7fee780cf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:40.608 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.627+0000 7fee7dc00640 1 --2- 172.21.15.204:0/1323993745 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee780cb850 0x7fee780cbc30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:40.608 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.627+0000 7fee7dc00640 1 -- 172.21.15.204:0/1323993745 >> 172.21.15.204:0/1323993745 conn(0x7fee7806e7d0 msgr2=0x7fee7806ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:59:40.609 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.627+0000 7fee7dc00640 1 -- 172.21.15.204:0/1323993745 shutdown_connections 2024-01-16T14:59:40.609 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.627+0000 7fee7dc00640 1 -- 172.21.15.204:0/1323993745 wait complete. 2024-01-16T14:59:40.609 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.629+0000 7fee7dc00640 1 Processor -- start 2024-01-16T14:59:40.609 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.629+0000 7fee7dc00640 1 -- start start 2024-01-16T14:59:40.612 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.630+0000 7fee7dc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee780b40d0 0x7fee780c4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:40.612 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.630+0000 7fee7dc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee780cb850 0x7fee7806bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:40.612 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.630+0000 7fee7dc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee780d0a00 0x7fee780b1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:40.612 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.630+0000 7fee7dc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7815e5c0 con 0x7fee780b40d0 2024-01-16T14:59:40.612 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.630+0000 7fee7dc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee780c5270 con 0x7fee780cb850 2024-01-16T14:59:40.612 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.630+0000 7fee7dc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee7815e000 con 0x7fee780d0a00 2024-01-16T14:59:40.612 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.630+0000 7fee77400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee780d0a00 0x7fee780b1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:40.612 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.630+0000 7fee77400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee780d0a00 0x7fee780b1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:47674/0 (socket says 172.21.15.204:47674) 2024-01-16T14:59:40.612 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.630+0000 7fee6fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee780cb850 0x7fee7806bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:40.612 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.630+0000 7fee77400640 1 -- 172.21.15.204:0/593436333 learned_addr learned my addr 172.21.15.204:0/593436333 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-01-16T14:59:40.612 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.630+0000 7fee76a00640 1 --2- 172.21.15.204:0/593436333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee780b40d0 0x7fee780c4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:40.612 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.630+0000 7fee77400640 1 -- 172.21.15.204:0/593436333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee780cb850 msgr2=0x7fee7806bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:40.612 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.630+0000 7fee77400640 1 --2- 172.21.15.204:0/593436333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee780cb850 0x7fee7806bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:40.613 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.630+0000 7fee77400640 1 -- 172.21.15.204:0/593436333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee780b40d0 msgr2=0x7fee780c4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:40.613 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.631+0000 7fee77400640 1 --2- 172.21.15.204:0/593436333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee780b40d0 0x7fee780c4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:40.613 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.631+0000 7fee77400640 1 -- 172.21.15.204:0/593436333 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee68009400 con 0x7fee780d0a00 2024-01-16T14:59:40.613 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.631+0000 7fee76a00640 1 --2- 172.21.15.204:0/593436333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee780b40d0 0x7fee780c4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:59:40.613 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.631+0000 7fee77400640 1 --2- 172.21.15.204:0/593436333 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee780d0a00 0x7fee780b1400 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fee70007e60 tx=0x7fee7000e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:40.613 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.631+0000 7fee6ea00640 1 -- 172.21.15.204:0/593436333 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee70014420 con 0x7fee780d0a00 2024-01-16T14:59:40.613 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.631+0000 7fee6ea00640 1 -- 172.21.15.204:0/593436333 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee700145a0 con 0x7fee780d0a00 2024-01-16T14:59:40.613 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.631+0000 7fee7dc00640 1 -- 172.21.15.204:0/593436333 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee7815e2e0 con 0x7fee780d0a00 2024-01-16T14:59:40.613 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.631+0000 7fee6ea00640 1 -- 172.21.15.204:0/593436333 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee70014950 con 0x7fee780d0a00 2024-01-16T14:59:40.613 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.631+0000 7fee7dc00640 1 -- 172.21.15.204:0/593436333 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee7806ba00 con 0x7fee780d0a00 2024-01-16T14:59:40.614 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.631+0000 7fee6ea00640 1 -- 172.21.15.204:0/593436333 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fee7001e460 con 0x7fee780d0a00 2024-01-16T14:59:40.614 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.632+0000 7fee6ea00640 1 -- 172.21.15.204:0/593436333 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fee7001eba0 con 0x7fee780d0a00 2024-01-16T14:59:40.614 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.632+0000 7fee7dc00640 1 -- 172.21.15.204:0/593436333 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee7804e8b0 con 0x7fee780d0a00 2024-01-16T14:59:40.617 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.637+0000 7fee6ea00640 1 -- 172.21.15.204:0/593436333 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fee7001edd0 con 0x7fee780d0a00 2024-01-16T14:59:40.623 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-16T14:59:40.643+0000 7feee7830040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-16T14:59:40.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.715+0000 7fee7dc00640 1 -- 172.21.15.204:0/593436333 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd new", "uuid": "73674129-a84e-4d31-8247-7d0179ae6df2", "id": 3} v 0) v1 -- 0x7fee780735d0 con 0x7fee780d0a00 2024-01-16T14:59:40.703 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.722+0000 7fee6ea00640 1 -- 172.21.15.204:0/593436333 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "73674129-a84e-4d31-8247-7d0179ae6df2", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fee7002d0b0 con 0x7fee780d0a00 2024-01-16T14:59:40.703 INFO:teuthology.orchestra.run.smithi204.stdout:3 2024-01-16T14:59:40.705 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.724+0000 7fee7dc00640 1 -- 172.21.15.204:0/593436333 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee780d0a00 msgr2=0x7fee780b1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:40.705 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.724+0000 7fee7dc00640 1 --2- 172.21.15.204:0/593436333 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee780d0a00 0x7fee780b1400 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fee70007e60 tx=0x7fee7000e4b0 comp rx=0 tx=0).stop 2024-01-16T14:59:40.705 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.724+0000 7fee7dc00640 1 -- 172.21.15.204:0/593436333 shutdown_connections 2024-01-16T14:59:40.705 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.724+0000 7fee7dc00640 1 --2- 172.21.15.204:0/593436333 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee780d0a00 0x7fee780b1400 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:40.705 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.724+0000 7fee7dc00640 1 --2- 172.21.15.204:0/593436333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee780cb850 0x7fee7806bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:40.705 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.724+0000 7fee7dc00640 1 --2- 172.21.15.204:0/593436333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee780b40d0 0x7fee780c4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:40.705 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.724+0000 7fee7dc00640 1 -- 172.21.15.204:0/593436333 >> 172.21.15.204:0/593436333 conn(0x7fee7806e7d0 msgr2=0x7fee780b45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:59:40.705 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.724+0000 7fee7dc00640 1 -- 172.21.15.204:0/593436333 shutdown_connections 2024-01-16T14:59:40.705 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.724+0000 7fee7dc00640 1 -- 172.21.15.204:0/593436333 wait complete. 2024-01-16T14:59:40.721 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new cea28c9b-2f1c-4087-b415-9960789131d6 4 2024-01-16T14:59:40.954 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.973+0000 7f2d24e00640 1 Processor -- start 2024-01-16T14:59:40.954 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.973+0000 7f2d24e00640 1 -- start start 2024-01-16T14:59:40.956 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.975+0000 7f2d24e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2d200cc0f0 0x7f2d200c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:40.956 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.975+0000 7f2d24e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2d200c57e0 0x7f2d2006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:40.956 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.975+0000 7f2d24e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d2015eea0 con 0x7f2d2015b440 2024-01-16T14:59:40.957 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.975+0000 7f2d24e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d2015b820 con 0x7f2d200c57e0 2024-01-16T14:59:40.957 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.975+0000 7f2d24e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d200abf30 con 0x7f2d200cc0f0 2024-01-16T14:59:40.957 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.975+0000 7f2d1de00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f2d2015b440 0x7f2d200cb9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:35952/0 (socket says 172.21.15.204:35952) 2024-01-16T14:59:40.957 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.975+0000 7f2d1de00640 1 -- 172.21.15.204:0/1470014306 learned_addr learned my addr 172.21.15.204:0/1470014306 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-01-16T14:59:40.957 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.975+0000 7f2d1d400640 1 --2- 172.21.15.204:0/1470014306 >> v2:172.21.15.204:3300/0 conn(0x7f2d200cc0f0 0x7f2d200c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:40.957 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.975+0000 7f2d1ca00640 1 -- 172.21.15.204:0/1470014306 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2760454352 0 0) 0x7f2d2015eea0 con 0x7f2d2015b440 2024-01-16T14:59:40.957 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.975+0000 7f2d1e800640 1 --2- 172.21.15.204:0/1470014306 >> v2:172.21.15.143:3300/0 conn(0x7f2d200c57e0 0x7f2d2006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:40.957 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.975+0000 7f2d1ca00640 1 -- 172.21.15.204:0/1470014306 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d00003520 con 0x7f2d2015b440 2024-01-16T14:59:40.957 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.975+0000 7f2d1d400640 1 -- 172.21.15.204:0/1470014306 >> v2:172.21.15.143:3300/0 conn(0x7f2d200c57e0 msgr2=0x7f2d2006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:59:40.957 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.975+0000 7f2d1d400640 1 --2- 172.21.15.204:0/1470014306 >> v2:172.21.15.143:3300/0 conn(0x7f2d200c57e0 0x7f2d2006c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:40.957 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.975+0000 7f2d1d400640 1 -- 172.21.15.204:0/1470014306 >> v1:172.21.15.204:6789/0 conn(0x7f2d2015b440 legacy=0x7f2d200cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:40.957 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.975+0000 7f2d1d400640 1 -- 172.21.15.204:0/1470014306 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2015f0f0 con 0x7f2d200cc0f0 2024-01-16T14:59:40.957 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.976+0000 7f2d1e800640 1 --2- 172.21.15.204:0/1470014306 >> v2:172.21.15.143:3300/0 conn(0x7f2d200c57e0 0x7f2d2006c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:59:40.958 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.977+0000 7f2d1d400640 1 --2- 172.21.15.204:0/1470014306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d200cc0f0 0x7f2d200c52a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2d2006e720 tx=0x7f2d0802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce97715d36ccc048 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:40.959 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.977+0000 7f2d1ca00640 1 -- 172.21.15.204:0/1470014306 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0802e070 con 0x7f2d200cc0f0 2024-01-16T14:59:40.959 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.977+0000 7f2d1ca00640 1 -- 172.21.15.204:0/1470014306 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d080315a0 con 0x7f2d200cc0f0 2024-01-16T14:59:40.959 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.977+0000 7f2d1ca00640 1 -- 172.21.15.204:0/1470014306 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d08031930 con 0x7f2d200cc0f0 2024-01-16T14:59:40.959 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.977+0000 7f2d24e00640 1 -- 172.21.15.204:0/1470014306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d200cc0f0 msgr2=0x7f2d200c52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:40.959 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.977+0000 7f2d24e00640 1 --2- 172.21.15.204:0/1470014306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d200cc0f0 0x7f2d200c52a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2d2006e720 tx=0x7f2d0802b840 comp rx=0 tx=0).stop 2024-01-16T14:59:40.959 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.978+0000 7f2d24e00640 1 -- 172.21.15.204:0/1470014306 shutdown_connections 2024-01-16T14:59:40.959 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.978+0000 7f2d24e00640 1 --2- 172.21.15.204:0/1470014306 >> v2:172.21.15.143:3300/0 conn(0x7f2d200c57e0 0x7f2d2006c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:40.959 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.978+0000 7f2d24e00640 1 --2- 172.21.15.204:0/1470014306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d200cc0f0 0x7f2d200c52a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:40.959 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.978+0000 7f2d24e00640 1 -- 172.21.15.204:0/1470014306 >> 172.21.15.204:0/1470014306 conn(0x7f2d200b0c50 msgr2=0x7f2d200b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:59:40.959 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.978+0000 7f2d24e00640 1 -- 172.21.15.204:0/1470014306 shutdown_connections 2024-01-16T14:59:40.959 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.978+0000 7f2d24e00640 1 -- 172.21.15.204:0/1470014306 wait complete. 2024-01-16T14:59:40.960 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.979+0000 7f2d24e00640 1 Processor -- start 2024-01-16T14:59:40.960 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.979+0000 7f2d24e00640 1 -- start start 2024-01-16T14:59:40.961 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.980+0000 7f2d24e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d200c57e0 0x7f2d2009bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:40.962 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.980+0000 7f2d24e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d200cc0f0 0x7f2d20145fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:40.962 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.980+0000 7f2d24e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d2015b440 0x7f2d20145630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:40.962 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.980+0000 7f2d24e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2015c300 con 0x7f2d200cc0f0 2024-01-16T14:59:40.962 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.980+0000 7f2d24e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d201543a0 con 0x7f2d2015b440 2024-01-16T14:59:40.962 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.980+0000 7f2d24e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d201464e0 con 0x7f2d200c57e0 2024-01-16T14:59:40.962 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.981+0000 7f2d1e800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d2015b440 0x7f2d20145630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:40.962 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.981+0000 7f2d1de00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d200c57e0 0x7f2d2009bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:40.962 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.981+0000 7f2d1e800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d2015b440 0x7f2d20145630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:44882/0 (socket says 172.21.15.204:44882) 2024-01-16T14:59:40.962 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.981+0000 7f2d1e800640 1 -- 172.21.15.204:0/106562863 learned_addr learned my addr 172.21.15.204:0/106562863 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-01-16T14:59:40.962 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.981+0000 7f2d1d400640 1 --2- 172.21.15.204:0/106562863 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d200cc0f0 0x7f2d20145fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:40.962 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.981+0000 7f2d1e800640 1 -- 172.21.15.204:0/106562863 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d200c57e0 msgr2=0x7f2d2009bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:40.962 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.981+0000 7f2d1e800640 1 --2- 172.21.15.204:0/106562863 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d200c57e0 0x7f2d2009bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:40.962 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.981+0000 7f2d1e800640 1 -- 172.21.15.204:0/106562863 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d200cc0f0 msgr2=0x7f2d20145fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:40.963 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.981+0000 7f2d1e800640 1 --2- 172.21.15.204:0/106562863 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d200cc0f0 0x7f2d20145fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:40.963 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.981+0000 7f2d1e800640 1 -- 172.21.15.204:0/106562863 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d08009400 con 0x7f2d2015b440 2024-01-16T14:59:40.964 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.981+0000 7f2d1e800640 1 --2- 172.21.15.204:0/106562863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d2015b440 0x7f2d20145630 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800cf20 tx=0x7f2d1800a7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:40.964 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.982+0000 7f2d16000640 1 -- 172.21.15.204:0/106562863 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1800d840 con 0x7f2d2015b440 2024-01-16T14:59:40.964 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.982+0000 7f2d16000640 1 -- 172.21.15.204:0/106562863 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d1800d9a0 con 0x7f2d2015b440 2024-01-16T14:59:40.964 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.982+0000 7f2d16000640 1 -- 172.21.15.204:0/106562863 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1800dd50 con 0x7f2d2015b440 2024-01-16T14:59:40.964 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.982+0000 7f2d24e00640 1 -- 172.21.15.204:0/106562863 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d201485a0 con 0x7f2d2015b440 2024-01-16T14:59:40.964 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.982+0000 7f2d24e00640 1 -- 172.21.15.204:0/106562863 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d20148b60 con 0x7f2d2015b440 2024-01-16T14:59:40.964 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.982+0000 7f2d16000640 1 -- 172.21.15.204:0/106562863 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2d1801f740 con 0x7f2d2015b440 2024-01-16T14:59:40.965 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.982+0000 7f2d16000640 1 -- 172.21.15.204:0/106562863 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f2d18015b80 con 0x7f2d2015b440 2024-01-16T14:59:40.965 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.983+0000 7f2d24e00640 1 -- 172.21.15.204:0/106562863 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d2004e8b0 con 0x7f2d2015b440 2024-01-16T14:59:40.966 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:40.986+0000 7f2d16000640 1 -- 172.21.15.204:0/106562863 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f2d1801a070 con 0x7f2d2015b440 2024-01-16T14:59:41.005 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T14:59:41.023+0000 7f27f9844040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-16T14:59:41.044 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.064+0000 7f2d24e00640 1 -- 172.21.15.204:0/106562863 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd new", "uuid": "cea28c9b-2f1c-4087-b415-9960789131d6", "id": 4} v 0) v1 -- 0x7f2d2006dc80 con 0x7f2d2015b440 2024-01-16T14:59:41.052 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.072+0000 7f2d16000640 1 -- 172.21.15.204:0/106562863 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cea28c9b-2f1c-4087-b415-9960789131d6", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2d18015e10 con 0x7f2d2015b440 2024-01-16T14:59:41.053 INFO:teuthology.orchestra.run.smithi204.stdout:4 2024-01-16T14:59:41.054 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.073+0000 7f2d24e00640 1 -- 172.21.15.204:0/106562863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d2015b440 msgr2=0x7f2d20145630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:41.054 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.073+0000 7f2d24e00640 1 --2- 172.21.15.204:0/106562863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d2015b440 0x7f2d20145630 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800cf20 tx=0x7f2d1800a7d0 comp rx=0 tx=0).stop 2024-01-16T14:59:41.054 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.073+0000 7f2d24e00640 1 -- 172.21.15.204:0/106562863 shutdown_connections 2024-01-16T14:59:41.054 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.073+0000 7f2d24e00640 1 --2- 172.21.15.204:0/106562863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d2015b440 0x7f2d20145630 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:41.054 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.073+0000 7f2d24e00640 1 --2- 172.21.15.204:0/106562863 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d200cc0f0 0x7f2d20145fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:41.054 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.073+0000 7f2d24e00640 1 --2- 172.21.15.204:0/106562863 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d200c57e0 0x7f2d2009bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:41.054 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.073+0000 7f2d24e00640 1 -- 172.21.15.204:0/106562863 >> 172.21.15.204:0/106562863 conn(0x7f2d200b0c50 msgr2=0x7f2d200cb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:59:41.054 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.073+0000 7f2d24e00640 1 -- 172.21.15.204:0/106562863 shutdown_connections 2024-01-16T14:59:41.054 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.073+0000 7f2d24e00640 1 -- 172.21.15.204:0/106562863 wait complete. 2024-01-16T14:59:41.069 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 33b63560-5574-4e78-983f-b398d67a2cb6 5 2024-01-16T14:59:41.101 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-16T14:59:41.121+0000 7feee7830040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-16T14:59:41.165 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T14:59:41.183+0000 7f27f9844040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-16T14:59:41.250 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.269+0000 7fe2c2800640 1 Processor -- start 2024-01-16T14:59:41.250 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.269+0000 7fe2c2800640 1 -- start start 2024-01-16T14:59:41.251 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.270+0000 7fe2c2800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe2bc0d1a00 0x7fe2bc0b2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:41.251 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.270+0000 7fe2c2800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2bc0b41b0 con 0x7fe2bc1609f0 2024-01-16T14:59:41.251 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.270+0000 7fe2c2800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2bc0b4490 con 0x7fe2bc160dd0 2024-01-16T14:59:41.251 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.270+0000 7fe2c2800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2bc0b0090 con 0x7fe2bc0d1a00 2024-01-16T14:59:41.251 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.270+0000 7fe2c0a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe2bc0d1a00 0x7fe2bc0b2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:41.251 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.270+0000 7fe2c1e00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fe2bc160dd0 0x7fe2bc0b3120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:35966/0 (socket says 172.21.15.204:35966) 2024-01-16T14:59:41.251 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.270+0000 7fe2c1e00640 1 -- 172.21.15.204:0/1411225508 learned_addr learned my addr 172.21.15.204:0/1411225508 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-01-16T14:59:41.252 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.271+0000 7fe2c0a00640 1 -- 172.21.15.204:0/1411225508 >> v1:172.21.15.204:6789/0 conn(0x7fe2bc160dd0 legacy=0x7fe2bc0b3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:41.252 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.271+0000 7fe2c0a00640 1 -- 172.21.15.204:0/1411225508 >> v1:172.21.15.143:6789/0 conn(0x7fe2bc1609f0 legacy=0x7fe2bc0d12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:41.252 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.271+0000 7fe2c0a00640 1 -- 172.21.15.204:0/1411225508 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2bc0b0370 con 0x7fe2bc0d1a00 2024-01-16T14:59:41.253 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.272+0000 7fe2c0a00640 1 --2- 172.21.15.204:0/1411225508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2bc0d1a00 0x7fe2bc0b2be0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe2a40097a0 tx=0x7fe2a402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2afa0916deee1508 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:41.253 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.272+0000 7fe2bbe00640 1 -- 172.21.15.204:0/1411225508 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4030610 con 0x7fe2bc0d1a00 2024-01-16T14:59:41.253 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.272+0000 7fe2bbe00640 1 -- 172.21.15.204:0/1411225508 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a4030770 con 0x7fe2bc0d1a00 2024-01-16T14:59:41.253 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.272+0000 7fe2bbe00640 1 -- 172.21.15.204:0/1411225508 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4030b00 con 0x7fe2bc0d1a00 2024-01-16T14:59:41.254 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.273+0000 7fe2c2800640 1 -- 172.21.15.204:0/1411225508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2bc0d1a00 msgr2=0x7fe2bc0b2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:41.254 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.273+0000 7fe2c2800640 1 --2- 172.21.15.204:0/1411225508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2bc0d1a00 0x7fe2bc0b2be0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe2a40097a0 tx=0x7fe2a402b960 comp rx=0 tx=0).stop 2024-01-16T14:59:41.254 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.273+0000 7fe2c2800640 1 -- 172.21.15.204:0/1411225508 shutdown_connections 2024-01-16T14:59:41.254 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.273+0000 7fe2c2800640 1 --2- 172.21.15.204:0/1411225508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2bc0d1a00 0x7fe2bc0b2be0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:41.254 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.273+0000 7fe2c2800640 1 -- 172.21.15.204:0/1411225508 >> 172.21.15.204:0/1411225508 conn(0x7fe2bc09c5b0 msgr2=0x7fe2bc0d3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:59:41.254 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.273+0000 7fe2c2800640 1 -- 172.21.15.204:0/1411225508 shutdown_connections 2024-01-16T14:59:41.254 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.273+0000 7fe2c2800640 1 -- 172.21.15.204:0/1411225508 wait complete. 2024-01-16T14:59:41.255 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.275+0000 7fe2c2800640 1 Processor -- start 2024-01-16T14:59:41.255 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.275+0000 7fe2c2800640 1 -- start start 2024-01-16T14:59:41.258 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.276+0000 7fe2c2800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2bc0d1a00 0x7fe2bc0c9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:41.258 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.276+0000 7fe2c2800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2bc1609f0 0x7fe2bc0c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:41.258 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.276+0000 7fe2c2800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2bc160dd0 0x7fe2bc0ce680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:41.258 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.276+0000 7fe2c2800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc0c7450 con 0x7fe2bc160dd0 2024-01-16T14:59:41.258 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.276+0000 7fe2c2800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc0cebc0 con 0x7fe2bc0d1a00 2024-01-16T14:59:41.258 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.276+0000 7fe2c2800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc0c96e0 con 0x7fe2bc1609f0 2024-01-16T14:59:41.259 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.276+0000 7fe2c1400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2bc0d1a00 0x7fe2bc0c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:41.259 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.276+0000 7fe2c0a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2bc1609f0 0x7fe2bc0c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:41.259 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.276+0000 7fe2c1400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2bc0d1a00 0x7fe2bc0c9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:44898/0 (socket says 172.21.15.204:44898) 2024-01-16T14:59:41.259 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.276+0000 7fe2c1400640 1 -- 172.21.15.204:0/1850575881 learned_addr learned my addr 172.21.15.204:0/1850575881 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-01-16T14:59:41.259 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.276+0000 7fe2c0a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2bc1609f0 0x7fe2bc0c6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:47700/0 (socket says 172.21.15.204:47700) 2024-01-16T14:59:41.259 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.276+0000 7fe2c1e00640 1 --2- 172.21.15.204:0/1850575881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2bc160dd0 0x7fe2bc0ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:41.259 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.277+0000 7fe2c1400640 1 -- 172.21.15.204:0/1850575881 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2bc1609f0 msgr2=0x7fe2bc0c6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:41.259 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.277+0000 7fe2c1400640 1 --2- 172.21.15.204:0/1850575881 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2bc1609f0 0x7fe2bc0c6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:41.259 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.277+0000 7fe2c1400640 1 -- 172.21.15.204:0/1850575881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2bc160dd0 msgr2=0x7fe2bc0ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:41.259 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.277+0000 7fe2c1400640 1 --2- 172.21.15.204:0/1850575881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2bc160dd0 0x7fe2bc0ce680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:41.259 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.277+0000 7fe2c1400640 1 -- 172.21.15.204:0/1850575881 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a4009400 con 0x7fe2bc0d1a00 2024-01-16T14:59:41.259 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.277+0000 7fe2c1400640 1 --2- 172.21.15.204:0/1850575881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2bc0d1a00 0x7fe2bc0c9100 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac00c450 tx=0x7fe2ac00c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:41.259 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.277+0000 7fe2b9600640 1 -- 172.21.15.204:0/1850575881 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac0052d0 con 0x7fe2bc0d1a00 2024-01-16T14:59:41.260 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.277+0000 7fe2b9600640 1 -- 172.21.15.204:0/1850575881 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2ac005430 con 0x7fe2bc0d1a00 2024-01-16T14:59:41.260 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.277+0000 7fe2b9600640 1 -- 172.21.15.204:0/1850575881 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac012870 con 0x7fe2bc0d1a00 2024-01-16T14:59:41.260 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.277+0000 7fe2c2800640 1 -- 172.21.15.204:0/1850575881 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2bc15ae20 con 0x7fe2bc0d1a00 2024-01-16T14:59:41.260 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.277+0000 7fe2c2800640 1 -- 172.21.15.204:0/1850575881 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2bc15b3e0 con 0x7fe2bc0d1a00 2024-01-16T14:59:41.260 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.278+0000 7fe2b9600640 1 -- 172.21.15.204:0/1850575881 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe2ac012cd0 con 0x7fe2bc0d1a00 2024-01-16T14:59:41.260 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.278+0000 7fe2b9600640 1 -- 172.21.15.204:0/1850575881 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fe2ac011380 con 0x7fe2bc0d1a00 2024-01-16T14:59:41.260 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.278+0000 7fe2c2800640 1 -- 172.21.15.204:0/1850575881 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2bc04e840 con 0x7fe2bc0d1a00 2024-01-16T14:59:41.261 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.281+0000 7fe2b9600640 1 -- 172.21.15.204:0/1850575881 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fe2ac0035f0 con 0x7fe2bc0d1a00 2024-01-16T14:59:41.270 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-16T14:59:41.290+0000 7feee7830040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-16T14:59:41.289 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T14:59:41.307+0000 7f27f9844040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-16T14:59:41.339 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.359+0000 7fe2c2800640 1 -- 172.21.15.204:0/1850575881 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd new", "uuid": "33b63560-5574-4e78-983f-b398d67a2cb6", "id": 5} v 0) v1 -- 0x7fe2bc06c540 con 0x7fe2bc0d1a00 2024-01-16T14:59:41.341 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T14:59:41.360+0000 7f27f9844040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-16T14:59:41.346 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.365+0000 7fe2b9600640 1 -- 172.21.15.204:0/1850575881 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "33b63560-5574-4e78-983f-b398d67a2cb6", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe2ac003790 con 0x7fe2bc0d1a00 2024-01-16T14:59:41.346 INFO:teuthology.orchestra.run.smithi204.stdout:5 2024-01-16T14:59:41.348 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.366+0000 7fe2c2800640 1 -- 172.21.15.204:0/1850575881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2bc0d1a00 msgr2=0x7fe2bc0c9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:41.348 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.367+0000 7fe2c2800640 1 --2- 172.21.15.204:0/1850575881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2bc0d1a00 0x7fe2bc0c9100 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac00c450 tx=0x7fe2ac00c910 comp rx=0 tx=0).stop 2024-01-16T14:59:41.348 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.367+0000 7fe2c2800640 1 -- 172.21.15.204:0/1850575881 shutdown_connections 2024-01-16T14:59:41.348 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.367+0000 7fe2c2800640 1 --2- 172.21.15.204:0/1850575881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2bc160dd0 0x7fe2bc0ce680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:41.348 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.367+0000 7fe2c2800640 1 --2- 172.21.15.204:0/1850575881 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2bc1609f0 0x7fe2bc0c6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:41.348 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.367+0000 7fe2c2800640 1 --2- 172.21.15.204:0/1850575881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2bc0d1a00 0x7fe2bc0c9100 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:41.348 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.367+0000 7fe2c2800640 1 -- 172.21.15.204:0/1850575881 >> 172.21.15.204:0/1850575881 conn(0x7fe2bc09c5b0 msgr2=0x7fe2bc15e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:59:41.348 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.367+0000 7fe2c2800640 1 -- 172.21.15.204:0/1850575881 shutdown_connections 2024-01-16T14:59:41.348 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.367+0000 7fe2c2800640 1 -- 172.21.15.204:0/1850575881 wait complete. 2024-01-16T14:59:41.364 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new df397427-8940-4eee-9c79-eece669d735d 6 2024-01-16T14:59:41.417 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-16T14:59:41.437+0000 7feee7830040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-16T14:59:41.450 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T14:59:41.469+0000 7f27f9844040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-16T14:59:41.474 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-16T14:59:41.493+0000 7feee7830040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-16T14:59:41.561 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T14:59:41.579+0000 7f27f9844040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-16T14:59:41.591 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-16T14:59:41.610+0000 7feee7830040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-16T14:59:41.597 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.616+0000 7f2657200640 1 Processor -- start 2024-01-16T14:59:41.597 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.616+0000 7f2657200640 1 -- start start 2024-01-16T14:59:41.599 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.618+0000 7f2657200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26581609f0 0x7f26580d12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:41.600 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.618+0000 7f2657200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f26580b2050 0x7f26580d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:41.600 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.618+0000 7f2657200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26580b41b0 con 0x7f2658160dd0 2024-01-16T14:59:41.600 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.618+0000 7f2657200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26580b4490 con 0x7f26580b2050 2024-01-16T14:59:41.600 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.618+0000 7f2657200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26580b0090 con 0x7f26581609f0 2024-01-16T14:59:41.600 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.618+0000 7f2655e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26581609f0 0x7f26580d12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:41.600 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.618+0000 7f2655e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26581609f0 0x7f26580d12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:47710/0 (socket says 172.21.15.204:47710) 2024-01-16T14:59:41.600 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.618+0000 7f2655e00640 1 -- 172.21.15.204:0/3801223562 learned_addr learned my addr 172.21.15.204:0/3801223562 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-01-16T14:59:41.600 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.618+0000 7f2656800640 1 --2- 172.21.15.204:0/3801223562 >> v2:172.21.15.143:3300/0 conn(0x7f26580b2050 0x7f26580d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:41.600 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.619+0000 7f2655e00640 1 -- 172.21.15.204:0/3801223562 >> v2:172.21.15.143:3300/0 conn(0x7f26580b2050 msgr2=0x7f26580d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:59:41.601 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.619+0000 7f2655e00640 1 --2- 172.21.15.204:0/3801223562 >> v2:172.21.15.143:3300/0 conn(0x7f26580b2050 0x7f26580d1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:41.601 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.619+0000 7f2655e00640 1 -- 172.21.15.204:0/3801223562 >> v1:172.21.15.143:6789/0 conn(0x7f2658160dd0 legacy=0x7f26580b0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:59:41.601 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.619+0000 7f2655e00640 1 -- 172.21.15.204:0/3801223562 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26580b0370 con 0x7f26581609f0 2024-01-16T14:59:41.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.620+0000 7f2655e00640 1 --2- 172.21.15.204:0/3801223562 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26581609f0 0x7f26580d12f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2648009870 tx=0x7f264802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94619c5bd73caa57 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:41.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.620+0000 7f2655400640 1 -- 172.21.15.204:0/3801223562 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2648031640 con 0x7f26581609f0 2024-01-16T14:59:41.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.620+0000 7f2655400640 1 -- 172.21.15.204:0/3801223562 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26480317a0 con 0x7f26581609f0 2024-01-16T14:59:41.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.620+0000 7f2655400640 1 -- 172.21.15.204:0/3801223562 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2648031b30 con 0x7f26581609f0 2024-01-16T14:59:41.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.620+0000 7f2657200640 1 -- 172.21.15.204:0/3801223562 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26581609f0 msgr2=0x7f26580d12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:41.603 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.620+0000 7f2657200640 1 --2- 172.21.15.204:0/3801223562 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26581609f0 0x7f26580d12f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2648009870 tx=0x7f264802b8f0 comp rx=0 tx=0).stop 2024-01-16T14:59:41.603 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.621+0000 7f2657200640 1 -- 172.21.15.204:0/3801223562 shutdown_connections 2024-01-16T14:59:41.603 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.621+0000 7f2657200640 1 --2- 172.21.15.204:0/3801223562 >> v2:172.21.15.143:3300/0 conn(0x7f26580b2050 0x7f26580d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:41.603 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.621+0000 7f2657200640 1 --2- 172.21.15.204:0/3801223562 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26581609f0 0x7f26580d12f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:41.603 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.621+0000 7f2657200640 1 -- 172.21.15.204:0/3801223562 >> 172.21.15.204:0/3801223562 conn(0x7f265809c5b0 msgr2=0x7f26580d3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:59:41.603 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.621+0000 7f2657200640 1 -- 172.21.15.204:0/3801223562 shutdown_connections 2024-01-16T14:59:41.603 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.621+0000 7f2657200640 1 -- 172.21.15.204:0/3801223562 wait complete. 2024-01-16T14:59:41.603 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.622+0000 7f2657200640 1 Processor -- start 2024-01-16T14:59:41.604 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.623+0000 7f2657200640 1 -- start start 2024-01-16T14:59:41.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.623+0000 7f2657200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26580b2050 0x7f26580cce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:41.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.623+0000 7f2657200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26581609f0 0x7f26580ca350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:41.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.624+0000 7f2657200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2658160dd0 0x7f265814e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:41.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.624+0000 7f2657200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26580cb6d0 con 0x7f2658160dd0 2024-01-16T14:59:41.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.624+0000 7f2657200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26580caa50 con 0x7f26580b2050 2024-01-16T14:59:41.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.624+0000 7f2657200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26580cad30 con 0x7f26581609f0 2024-01-16T14:59:41.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.624+0000 7f2655e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26580b2050 0x7f26580cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:41.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.624+0000 7f2655e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26580b2050 0x7f26580cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:44910/0 (socket says 172.21.15.204:44910) 2024-01-16T14:59:41.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.624+0000 7f2655e00640 1 -- 172.21.15.204:0/1598390649 learned_addr learned my addr 172.21.15.204:0/1598390649 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-01-16T14:59:41.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.624+0000 7f264fe00640 1 --2- 172.21.15.204:0/1598390649 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26581609f0 0x7f26580ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:41.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.624+0000 7f2656800640 1 --2- 172.21.15.204:0/1598390649 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2658160dd0 0x7f265814e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:41.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.624+0000 7f2655e00640 1 -- 172.21.15.204:0/1598390649 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26581609f0 msgr2=0x7f26580ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:41.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.624+0000 7f2655e00640 1 --2- 172.21.15.204:0/1598390649 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26581609f0 0x7f26580ca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:41.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.624+0000 7f2655e00640 1 -- 172.21.15.204:0/1598390649 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2658160dd0 msgr2=0x7f265814e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:41.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.624+0000 7f2655e00640 1 --2- 172.21.15.204:0/1598390649 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2658160dd0 0x7f265814e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:41.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.624+0000 7f2655e00640 1 -- 172.21.15.204:0/1598390649 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26480094d0 con 0x7f26580b2050 2024-01-16T14:59:41.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.624+0000 7f2655e00640 1 --2- 172.21.15.204:0/1598390649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26580b2050 0x7f26580cce60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2648000c00 tx=0x7f26480046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:41.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.625+0000 7f264e000640 1 -- 172.21.15.204:0/1598390649 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264803ac60 con 0x7f26580b2050 2024-01-16T14:59:41.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.625+0000 7f264e000640 1 -- 172.21.15.204:0/1598390649 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f264803ade0 con 0x7f26580b2050 2024-01-16T14:59:41.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.625+0000 7f264e000640 1 -- 172.21.15.204:0/1598390649 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2648044630 con 0x7f26580b2050 2024-01-16T14:59:41.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.625+0000 7f2657200640 1 -- 172.21.15.204:0/1598390649 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26580cb010 con 0x7f26580b2050 2024-01-16T14:59:41.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.625+0000 7f2657200640 1 -- 172.21.15.204:0/1598390649 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f265814eae0 con 0x7f26580b2050 2024-01-16T14:59:41.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.625+0000 7f264e000640 1 -- 172.21.15.204:0/1598390649 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f264802f050 con 0x7f26580b2050 2024-01-16T14:59:41.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.625+0000 7f264e000640 1 -- 172.21.15.204:0/1598390649 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f2648044a90 con 0x7f26580b2050 2024-01-16T14:59:41.608 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.626+0000 7f2657200640 1 -- 172.21.15.204:0/1598390649 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26580cb810 con 0x7f26580b2050 2024-01-16T14:59:41.609 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.629+0000 7f264e000640 1 -- 172.21.15.204:0/1598390649 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f26580cb810 con 0x7f26580b2050 2024-01-16T14:59:41.687 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.707+0000 7f2657200640 1 -- 172.21.15.204:0/1598390649 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd new", "uuid": "df397427-8940-4eee-9c79-eece669d735d", "id": 6} v 0) v1 -- 0x7f265804e840 con 0x7f26580b2050 2024-01-16T14:59:41.694 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.714+0000 7f264e000640 1 -- 172.21.15.204:0/1598390649 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "df397427-8940-4eee-9c79-eece669d735d", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2648052070 con 0x7f26580b2050 2024-01-16T14:59:41.694 INFO:teuthology.orchestra.run.smithi204.stdout:6 2024-01-16T14:59:41.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.715+0000 7f2657200640 1 -- 172.21.15.204:0/1598390649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26580b2050 msgr2=0x7f26580cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:41.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.715+0000 7f2657200640 1 --2- 172.21.15.204:0/1598390649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26580b2050 0x7f26580cce60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2648000c00 tx=0x7f26480046c0 comp rx=0 tx=0).stop 2024-01-16T14:59:41.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.715+0000 7f2657200640 1 -- 172.21.15.204:0/1598390649 shutdown_connections 2024-01-16T14:59:41.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.715+0000 7f2657200640 1 --2- 172.21.15.204:0/1598390649 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2658160dd0 0x7f265814e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:41.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.715+0000 7f2657200640 1 --2- 172.21.15.204:0/1598390649 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26581609f0 0x7f26580ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:41.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.715+0000 7f2657200640 1 --2- 172.21.15.204:0/1598390649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26580b2050 0x7f26580cce60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:41.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.715+0000 7f2657200640 1 -- 172.21.15.204:0/1598390649 >> 172.21.15.204:0/1598390649 conn(0x7f265809c5b0 msgr2=0x7f265806b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:59:41.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.715+0000 7f2657200640 1 -- 172.21.15.204:0/1598390649 shutdown_connections 2024-01-16T14:59:41.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.715+0000 7f2657200640 1 -- 172.21.15.204:0/1598390649 wait complete. 2024-01-16T14:59:41.708 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-16T14:59:41.728+0000 7feee7830040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-16T14:59:41.713 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new a2e5a15c-8312-405a-ac02-f7e1fc9df2e4 7 2024-01-16T14:59:41.721 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T14:59:41.739+0000 7f27f9844040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-16T14:59:41.769 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T14:59:41.788+0000 7f27f9844040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-16T14:59:41.876 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-16T14:59:41.895+0000 7feee7830040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-16T14:59:41.927 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-16T14:59:41.946+0000 7feee7830040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-16T14:59:41.928 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T14:59:41.947+0000 7f27f9844040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-16T14:59:41.944 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.963+0000 7f1a33400640 1 Processor -- start 2024-01-16T14:59:41.944 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.964+0000 7f1a33400640 1 -- start start 2024-01-16T14:59:41.948 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.965+0000 7f1a33400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1a2c0d1550 0x7f1a2c0d2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:41.948 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.965+0000 7f1a33400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1a2c0d3500 0x7f1a2c0ce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:41.948 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.965+0000 7f1a33400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a2c0d1ae0 con 0x7f1a2c160a00 2024-01-16T14:59:41.948 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.965+0000 7f1a33400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a2c0d1dc0 con 0x7f1a2c0d3500 2024-01-16T14:59:41.948 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.965+0000 7f1a33400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a2c0d20a0 con 0x7f1a2c0d1550 2024-01-16T14:59:41.949 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.966+0000 7f1a31600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1a2c0d1550 0x7f1a2c0d2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:41.949 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.966+0000 7f1a31600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1a2c0d1550 0x7f1a2c0d2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:51988/0 (socket says 172.21.15.204:51988) 2024-01-16T14:59:41.949 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.966+0000 7f1a31600640 1 -- 172.21.15.204:0/3667520924 learned_addr learned my addr 172.21.15.204:0/3667520924 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-01-16T14:59:41.949 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.966+0000 7f1a32a00640 1 --2- 172.21.15.204:0/3667520924 >> v2:172.21.15.143:3300/0 conn(0x7f1a2c0d3500 0x7f1a2c0ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:41.949 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.966+0000 7f1a31600640 1 -- 172.21.15.204:0/3667520924 >> v2:172.21.15.143:3300/0 conn(0x7f1a2c0d3500 msgr2=0x7f1a2c0ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:59:41.949 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.966+0000 7f1a31600640 1 --2- 172.21.15.204:0/3667520924 >> v2:172.21.15.143:3300/0 conn(0x7f1a2c0d3500 0x7f1a2c0ce660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:41.949 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.966+0000 7f1a31600640 1 -- 172.21.15.204:0/3667520924 >> v1:172.21.15.143:6789/0 conn(0x7f1a2c160a00 legacy=0x7f1a2c0d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:59:41.949 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.966+0000 7f1a31600640 1 -- 172.21.15.204:0/3667520924 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a2c0ce140 con 0x7f1a2c0d1550 2024-01-16T14:59:41.949 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.967+0000 7f1a31600640 1 --2- 172.21.15.204:0/3667520924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a2c0d1550 0x7f1a2c0d2fc0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c08b910 tx=0x7f1a1402b7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed94da4458bbeb23 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:41.949 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.967+0000 7f1a30c00640 1 -- 172.21.15.204:0/3667520924 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a14031750 con 0x7f1a2c0d1550 2024-01-16T14:59:41.949 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.967+0000 7f1a30c00640 1 -- 172.21.15.204:0/3667520924 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a140318b0 con 0x7f1a2c0d1550 2024-01-16T14:59:41.949 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.967+0000 7f1a30c00640 1 -- 172.21.15.204:0/3667520924 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a14031c40 con 0x7f1a2c0d1550 2024-01-16T14:59:41.950 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.968+0000 7f1a33400640 1 -- 172.21.15.204:0/3667520924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a2c0d1550 msgr2=0x7f1a2c0d2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:41.951 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.968+0000 7f1a33400640 1 --2- 172.21.15.204:0/3667520924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a2c0d1550 0x7f1a2c0d2fc0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c08b910 tx=0x7f1a1402b7f0 comp rx=0 tx=0).stop 2024-01-16T14:59:41.951 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.968+0000 7f1a33400640 1 -- 172.21.15.204:0/3667520924 shutdown_connections 2024-01-16T14:59:41.951 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.968+0000 7f1a33400640 1 --2- 172.21.15.204:0/3667520924 >> v2:172.21.15.143:3300/0 conn(0x7f1a2c0d3500 0x7f1a2c0ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:41.951 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.968+0000 7f1a33400640 1 --2- 172.21.15.204:0/3667520924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a2c0d1550 0x7f1a2c0d2fc0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:41.951 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.968+0000 7f1a33400640 1 -- 172.21.15.204:0/3667520924 >> 172.21.15.204:0/3667520924 conn(0x7f1a2c08cc90 msgr2=0x7f1a2c09c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:59:41.951 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.968+0000 7f1a33400640 1 -- 172.21.15.204:0/3667520924 shutdown_connections 2024-01-16T14:59:41.951 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.968+0000 7f1a33400640 1 -- 172.21.15.204:0/3667520924 wait complete. 2024-01-16T14:59:41.951 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.970+0000 7f1a33400640 1 Processor -- start 2024-01-16T14:59:41.951 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.970+0000 7f1a33400640 1 -- start start 2024-01-16T14:59:41.954 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.971+0000 7f1a33400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a2c0d1550 0x7f1a2c0b79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:41.954 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.971+0000 7f1a33400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a2c0d3500 0x7f1a2c06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:41.954 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.971+0000 7f1a33400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a2c160a00 0x7f1a2c06c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:41.954 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.971+0000 7f1a33400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c0b5f30 con 0x7f1a2c160a00 2024-01-16T14:59:41.954 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.971+0000 7f1a33400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c0b6210 con 0x7f1a2c0d1550 2024-01-16T14:59:41.954 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.971+0000 7f1a33400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c0b64f0 con 0x7f1a2c0d3500 2024-01-16T14:59:41.954 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.971+0000 7f1a32000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a2c0d1550 0x7f1a2c0b79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:41.954 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.971+0000 7f1a31600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a2c0d3500 0x7f1a2c06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:41.954 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.971+0000 7f1a31600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a2c0d3500 0x7f1a2c06bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:48382/0 (socket says 172.21.15.204:48382) 2024-01-16T14:59:41.954 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.971+0000 7f1a31600640 1 -- 172.21.15.204:0/4133552392 learned_addr learned my addr 172.21.15.204:0/4133552392 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-01-16T14:59:41.954 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.972+0000 7f1a32a00640 1 --2- 172.21.15.204:0/4133552392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a2c160a00 0x7f1a2c06c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:41.954 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.972+0000 7f1a32000640 1 -- 172.21.15.204:0/4133552392 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a2c0d3500 msgr2=0x7f1a2c06bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:41.954 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.972+0000 7f1a32000640 1 --2- 172.21.15.204:0/4133552392 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a2c0d3500 0x7f1a2c06bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:41.955 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.972+0000 7f1a32000640 1 -- 172.21.15.204:0/4133552392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a2c160a00 msgr2=0x7f1a2c06c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:41.955 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.972+0000 7f1a32000640 1 --2- 172.21.15.204:0/4133552392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a2c160a00 0x7f1a2c06c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:41.955 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.972+0000 7f1a32000640 1 -- 172.21.15.204:0/4133552392 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a140094d0 con 0x7f1a2c0d1550 2024-01-16T14:59:41.955 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.972+0000 7f1a32a00640 1 --2- 172.21.15.204:0/4133552392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a2c160a00 0x7f1a2c06c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:59:41.955 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.972+0000 7f1a31600640 1 --2- 172.21.15.204:0/4133552392 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a2c0d3500 0x7f1a2c06bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:59:41.955 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.972+0000 7f1a32000640 1 --2- 172.21.15.204:0/4133552392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a2c0d1550 0x7f1a2c0b79e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c00b3e0 tx=0x7f1a1c00b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:41.955 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.972+0000 7f1a2a000640 1 -- 172.21.15.204:0/4133552392 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c004020 con 0x7f1a2c0d1550 2024-01-16T14:59:41.955 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.972+0000 7f1a2a000640 1 -- 172.21.15.204:0/4133552392 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a1c004180 con 0x7f1a2c0d1550 2024-01-16T14:59:41.955 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.972+0000 7f1a2a000640 1 -- 172.21.15.204:0/4133552392 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c011630 con 0x7f1a2c0d1550 2024-01-16T14:59:41.955 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.972+0000 7f1a33400640 1 -- 172.21.15.204:0/4133552392 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a2c0b67d0 con 0x7f1a2c0d1550 2024-01-16T14:59:41.955 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.973+0000 7f1a33400640 1 -- 172.21.15.204:0/4133552392 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a2c0b8710 con 0x7f1a2c0d1550 2024-01-16T14:59:41.955 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.973+0000 7f1a2a000640 1 -- 172.21.15.204:0/4133552392 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1a1c011a90 con 0x7f1a2c0d1550 2024-01-16T14:59:41.955 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.973+0000 7f1a2a000640 1 -- 172.21.15.204:0/4133552392 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f1a1c011d40 con 0x7f1a2c0d1550 2024-01-16T14:59:41.956 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.974+0000 7f1a33400640 1 -- 172.21.15.204:0/4133552392 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a2c04e840 con 0x7f1a2c0d1550 2024-01-16T14:59:41.956 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:41.976+0000 7f1a2a000640 1 -- 172.21.15.204:0/4133552392 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f1a1c015030 con 0x7f1a2c0d1550 2024-01-16T14:59:42.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:42.054+0000 7f1a33400640 1 -- 172.21.15.204:0/4133552392 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a2e5a15c-8312-405a-ac02-f7e1fc9df2e4", "id": 7} v 0) v1 -- 0x7f1a2c0b8ca0 con 0x7f1a2c0d1550 2024-01-16T14:59:42.042 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:42.061+0000 7f1a2a000640 1 -- 172.21.15.204:0/4133552392 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a2e5a15c-8312-405a-ac02-f7e1fc9df2e4", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1a1c003d20 con 0x7f1a2c0d1550 2024-01-16T14:59:42.042 INFO:teuthology.orchestra.run.smithi204.stdout:7 2024-01-16T14:59:42.044 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:42.063+0000 7f1a33400640 1 -- 172.21.15.204:0/4133552392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a2c0d1550 msgr2=0x7f1a2c0b79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:42.044 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:42.063+0000 7f1a33400640 1 --2- 172.21.15.204:0/4133552392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a2c0d1550 0x7f1a2c0b79e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c00b3e0 tx=0x7f1a1c00b8a0 comp rx=0 tx=0).stop 2024-01-16T14:59:42.044 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:42.063+0000 7f1a33400640 1 -- 172.21.15.204:0/4133552392 shutdown_connections 2024-01-16T14:59:42.044 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:42.063+0000 7f1a33400640 1 --2- 172.21.15.204:0/4133552392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a2c160a00 0x7f1a2c06c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:42.044 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:42.063+0000 7f1a33400640 1 --2- 172.21.15.204:0/4133552392 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a2c0d3500 0x7f1a2c06bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:42.045 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:42.063+0000 7f1a33400640 1 --2- 172.21.15.204:0/4133552392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a2c0d1550 0x7f1a2c0b79e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:42.045 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:42.063+0000 7f1a33400640 1 -- 172.21.15.204:0/4133552392 >> 172.21.15.204:0/4133552392 conn(0x7f1a2c08cc90 msgr2=0x7f1a2c15e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:59:42.045 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:42.063+0000 7f1a33400640 1 -- 172.21.15.204:0/4133552392 shutdown_connections 2024-01-16T14:59:42.045 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-16T14:59:42.063+0000 7f1a33400640 1 -- 172.21.15.204:0/4133552392 wait complete. 2024-01-16T14:59:42.061 INFO:tasks.ceph.osd.0:Restarting daemon 2024-01-16T14:59:42.061 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-01-16T14:59:42.068 INFO:tasks.ceph.osd.0:Started 2024-01-16T14:59:42.068 INFO:tasks.ceph.osd.1:Restarting daemon 2024-01-16T14:59:42.068 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-01-16T14:59:42.077 INFO:tasks.ceph.osd.1:Started 2024-01-16T14:59:42.077 INFO:tasks.ceph.osd.2:Restarting daemon 2024-01-16T14:59:42.078 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-01-16T14:59:42.083 INFO:tasks.ceph.osd.2:Started 2024-01-16T14:59:42.083 INFO:tasks.ceph.osd.3:Restarting daemon 2024-01-16T14:59:42.083 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-01-16T14:59:42.088 INFO:tasks.ceph.osd.3:Started 2024-01-16T14:59:42.088 INFO:tasks.ceph.osd.4:Restarting daemon 2024-01-16T14:59:42.088 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-01-16T14:59:42.095 INFO:tasks.ceph.osd.4:Started 2024-01-16T14:59:42.095 INFO:tasks.ceph.osd.5:Restarting daemon 2024-01-16T14:59:42.095 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-01-16T14:59:42.096 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-16T14:59:42.115+0000 7feee7830040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-16T14:59:42.102 INFO:tasks.ceph.osd.5:Started 2024-01-16T14:59:42.102 INFO:tasks.ceph.osd.6:Restarting daemon 2024-01-16T14:59:42.102 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-01-16T14:59:42.107 INFO:tasks.ceph.osd.6:Started 2024-01-16T14:59:42.107 INFO:tasks.ceph.osd.7:Restarting daemon 2024-01-16T14:59:42.107 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-01-16T14:59:42.112 INFO:tasks.ceph.osd.7:Started 2024-01-16T14:59:42.112 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T14:59:42.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.370+0000 7f1d52400640 1 Processor -- start 2024-01-16T14:59:42.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.371+0000 7f1d52400640 1 -- start start 2024-01-16T14:59:42.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.372+0000 7f1d52400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1d4c0cf5d0 0x7f1d4c06d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:42.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.372+0000 7f1d52400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d4c15e8b0 con 0x7f1d4c15b320 2024-01-16T14:59:42.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.372+0000 7f1d52400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d4c070180 con 0x7f1d4c15af40 2024-01-16T14:59:42.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.372+0000 7f1d52400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c15eb90 con 0x7f1d4c0cf5d0 2024-01-16T14:59:42.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.372+0000 7f1d4be00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f1d4c15b320 0x7f1d4c06b9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51658/0 (socket says 172.21.15.143:51658) 2024-01-16T14:59:42.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.372+0000 7f1d4be00640 1 -- 172.21.15.143:0/3107636539 learned_addr learned my addr 172.21.15.143:0/3107636539 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T14:59:42.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.372+0000 7f1d4aa00640 1 --2- 172.21.15.143:0/3107636539 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1d4c0cf5d0 0x7f1d4c06d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:42.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.372+0000 7f1d4a000640 1 -- 172.21.15.143:0/3107636539 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 493406599 0 0) 0x7f1d4c15e8b0 con 0x7f1d4c15b320 2024-01-16T14:59:42.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.372+0000 7f1d4a000640 1 -- 172.21.15.143:0/3107636539 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d2c003520 con 0x7f1d4c15b320 2024-01-16T14:59:42.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.373+0000 7f1d4a000640 1 -- 172.21.15.143:0/3107636539 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3761157867 0 0) 0x7f1d4c070180 con 0x7f1d4c15af40 2024-01-16T14:59:42.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.373+0000 7f1d4a000640 1 -- 172.21.15.143:0/3107636539 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d4c15e8b0 con 0x7f1d4c15af40 2024-01-16T14:59:42.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.373+0000 7f1d4aa00640 1 -- 172.21.15.143:0/3107636539 >> v1:172.21.15.204:6789/0 conn(0x7f1d4c15af40 legacy=0x7f1d4c0ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:42.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.373+0000 7f1d4aa00640 1 -- 172.21.15.143:0/3107636539 >> v1:172.21.15.143:6789/0 conn(0x7f1d4c15b320 legacy=0x7f1d4c06b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:42.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.373+0000 7f1d4aa00640 1 -- 172.21.15.143:0/3107636539 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d4c070460 con 0x7f1d4c0cf5d0 2024-01-16T14:59:42.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.374+0000 7f1d4aa00640 1 --2- 172.21.15.143:0/3107636539 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1d4c0cf5d0 0x7f1d4c06d220 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1d340097d0 tx=0x7f1d3402b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d5060e9fad61e23 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:42.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.374+0000 7f1d4a000640 1 -- 172.21.15.143:0/3107636539 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d34031890 con 0x7f1d4c0cf5d0 2024-01-16T14:59:42.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.374+0000 7f1d4a000640 1 -- 172.21.15.143:0/3107636539 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d340319f0 con 0x7f1d4c0cf5d0 2024-01-16T14:59:42.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.374+0000 7f1d4a000640 1 -- 172.21.15.143:0/3107636539 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d34031d80 con 0x7f1d4c0cf5d0 2024-01-16T14:59:42.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.374+0000 7f1d52400640 1 -- 172.21.15.143:0/3107636539 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1d4c0cf5d0 msgr2=0x7f1d4c06d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:42.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.374+0000 7f1d52400640 1 --2- 172.21.15.143:0/3107636539 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1d4c0cf5d0 0x7f1d4c06d220 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1d340097d0 tx=0x7f1d3402b930 comp rx=0 tx=0).stop 2024-01-16T14:59:42.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.375+0000 7f1d52400640 1 -- 172.21.15.143:0/3107636539 shutdown_connections 2024-01-16T14:59:42.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.375+0000 7f1d52400640 1 --2- 172.21.15.143:0/3107636539 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1d4c0cf5d0 0x7f1d4c06d220 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:42.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.375+0000 7f1d52400640 1 -- 172.21.15.143:0/3107636539 >> 172.21.15.143:0/3107636539 conn(0x7f1d4c06e7f0 msgr2=0x7f1d4c06ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:59:42.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.375+0000 7f1d52400640 1 -- 172.21.15.143:0/3107636539 shutdown_connections 2024-01-16T14:59:42.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.375+0000 7f1d52400640 1 -- 172.21.15.143:0/3107636539 wait complete. 2024-01-16T14:59:42.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.376+0000 7f1d52400640 1 Processor -- start 2024-01-16T14:59:42.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.376+0000 7f1d52400640 1 -- start start 2024-01-16T14:59:42.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.377+0000 7f1d52400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d4c0cf5d0 0x7f1d4c0b6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:42.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.377+0000 7f1d52400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1d4c15af40 0x7f1d4c0b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:42.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.377+0000 7f1d52400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d4c15b320 0x7f1d4c0b2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:42.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.377+0000 7f1d52400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c0b5670 con 0x7f1d4c15b320 2024-01-16T14:59:42.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.377+0000 7f1d52400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c0b4dd0 con 0x7f1d4c0cf5d0 2024-01-16T14:59:42.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.377+0000 7f1d52400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c0b50b0 con 0x7f1d4c15af40 2024-01-16T14:59:42.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.378+0000 7f1d4be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d4c15b320 0x7f1d4c0b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:42.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.378+0000 7f1d4b400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d4c0cf5d0 0x7f1d4c0b6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:42.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.378+0000 7f1d4be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d4c15b320 0x7f1d4c0b2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52686/0 (socket says 172.21.15.143:52686) 2024-01-16T14:59:42.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.378+0000 7f1d4be00640 1 -- 172.21.15.143:0/9651658 learned_addr learned my addr 172.21.15.143:0/9651658 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T14:59:42.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.378+0000 7f1d4aa00640 1 --2- 172.21.15.143:0/9651658 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1d4c15af40 0x7f1d4c0b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:42.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.378+0000 7f1d4be00640 1 -- 172.21.15.143:0/9651658 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1d4c15af40 msgr2=0x7f1d4c0b43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:42.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.378+0000 7f1d4be00640 1 --2- 172.21.15.143:0/9651658 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1d4c15af40 0x7f1d4c0b43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:42.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.378+0000 7f1d4be00640 1 -- 172.21.15.143:0/9651658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d4c0cf5d0 msgr2=0x7f1d4c0b6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:42.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.378+0000 7f1d4be00640 1 --2- 172.21.15.143:0/9651658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d4c0cf5d0 0x7f1d4c0b6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:42.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.378+0000 7f1d4be00640 1 -- 172.21.15.143:0/9651658 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d34009400 con 0x7f1d4c15b320 2024-01-16T14:59:42.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.378+0000 7f1d4b400640 1 --2- 172.21.15.143:0/9651658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d4c0cf5d0 0x7f1d4c0b6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:59:42.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.378+0000 7f1d4be00640 1 --2- 172.21.15.143:0/9651658 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d4c15b320 0x7f1d4c0b2290 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1d4400caa0 tx=0x7f1d4400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:42.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.378+0000 7f1d43400640 1 -- 172.21.15.143:0/9651658 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4400ac00 con 0x7f1d4c15b320 2024-01-16T14:59:42.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.379+0000 7f1d43400640 1 -- 172.21.15.143:0/9651658 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d4400ad80 con 0x7f1d4c15b320 2024-01-16T14:59:42.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.379+0000 7f1d43400640 1 -- 172.21.15.143:0/9651658 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d44012640 con 0x7f1d4c15b320 2024-01-16T14:59:42.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.379+0000 7f1d52400640 1 -- 172.21.15.143:0/9651658 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d4c0b5390 con 0x7f1d4c15b320 2024-01-16T14:59:42.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.379+0000 7f1d52400640 1 -- 172.21.15.143:0/9651658 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d4c0b6c60 con 0x7f1d4c15b320 2024-01-16T14:59:42.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.379+0000 7f1d43400640 1 -- 172.21.15.143:0/9651658 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1d44012aa0 con 0x7f1d4c15b320 2024-01-16T14:59:42.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.379+0000 7f1d43400640 1 -- 172.21.15.143:0/9651658 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f1d44012d50 con 0x7f1d4c15b320 2024-01-16T14:59:42.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.380+0000 7f1d52400640 1 -- 172.21.15.143:0/9651658 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d4c1f4940 con 0x7f1d4c15b320 2024-01-16T14:59:42.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.382+0000 7f1d43400640 1 -- 172.21.15.143:0/9651658 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f1d4c1f4940 con 0x7f1d4c15b320 2024-01-16T14:59:42.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.445+0000 7f1d52400640 1 -- 172.21.15.143:0/9651658 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d4c0b6eb0 con 0x7f1d4c15b320 2024-01-16T14:59:42.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.446+0000 7f1d43400640 1 -- 172.21.15.143:0/9651658 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f1d44017070 con 0x7f1d4c15b320 2024-01-16T14:59:42.428 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:59:42.428 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":10,"fsid":"c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0","created":"2024-01-16T14:59:38.532601+0000","modified":"2024-01-16T14:59:42.056297+0000","last_up_change":"0.000000","last_in_change":"2024-01-16T14:59:42.056297+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f4c8891d-ab29-472a-907d-8446f20f7012","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c4a2788d-8b89-4992-82b2-7f4b9debd509","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"64645c1d-31b2-4794-b0f2-7ccf8df30393","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"73674129-a84e-4d31-8247-7d0179ae6df2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"cea28c9b-2f1c-4087-b415-9960789131d6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"33b63560-5574-4e78-983f-b398d67a2cb6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"df397427-8940-4eee-9c79-eece669d735d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a2e5a15c-8312-405a-ac02-f7e1fc9df2e4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T14:59:42.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.447+0000 7f1d52400640 1 -- 172.21.15.143:0/9651658 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d4c15b320 msgr2=0x7f1d4c0b2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:42.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.447+0000 7f1d52400640 1 --2- 172.21.15.143:0/9651658 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d4c15b320 0x7f1d4c0b2290 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1d4400caa0 tx=0x7f1d4400cf60 comp rx=0 tx=0).stop 2024-01-16T14:59:42.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.448+0000 7f1d52400640 1 -- 172.21.15.143:0/9651658 shutdown_connections 2024-01-16T14:59:42.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.448+0000 7f1d52400640 1 --2- 172.21.15.143:0/9651658 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d4c15b320 0x7f1d4c0b2290 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:42.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.448+0000 7f1d52400640 1 --2- 172.21.15.143:0/9651658 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1d4c15af40 0x7f1d4c0b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:42.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.448+0000 7f1d52400640 1 --2- 172.21.15.143:0/9651658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d4c0cf5d0 0x7f1d4c0b6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:42.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.448+0000 7f1d52400640 1 -- 172.21.15.143:0/9651658 >> 172.21.15.143:0/9651658 conn(0x7f1d4c06e7f0 msgr2=0x7f1d4c15be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:59:42.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.448+0000 7f1d52400640 1 -- 172.21.15.143:0/9651658 shutdown_connections 2024-01-16T14:59:42.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:42.448+0000 7f1d52400640 1 -- 172.21.15.143:0/9651658 wait complete. 2024-01-16T14:59:42.446 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-01-16T14:59:42.446 INFO:tasks.ceph:Waiting for OSDs to come up 2024-01-16T14:59:42.786 INFO:tasks.ceph.osd.0.smithi143.stderr:2024-01-16T14:59:42.804+0000 7fce92822500 -1 Falling back to public interface 2024-01-16T14:59:42.786 INFO:tasks.ceph.osd.1.smithi143.stderr:2024-01-16T14:59:42.805+0000 7fed59994500 -1 Falling back to public interface 2024-01-16T14:59:42.786 INFO:tasks.ceph.osd.1.smithi143.stderr:2024-01-16T14:59:42.805+0000 7fed58400640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6801/0: (98) Address already in use 2024-01-16T14:59:42.789 INFO:tasks.ceph.osd.0.smithi143.stderr:2024-01-16T14:59:42.808+0000 7fce90e00640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6811/0: (98) Address already in use 2024-01-16T14:59:42.794 INFO:tasks.ceph.osd.2.smithi143.stderr:2024-01-16T14:59:42.813+0000 7f5a9c202500 -1 Falling back to public interface 2024-01-16T14:59:42.794 INFO:tasks.ceph.osd.6.smithi204.stderr:2024-01-16T14:59:42.814+0000 7f0596e12500 -1 Falling back to public interface 2024-01-16T14:59:42.797 INFO:tasks.ceph.osd.5.smithi204.stderr:2024-01-16T14:59:42.816+0000 7f8c6dcfa500 -1 Falling back to public interface 2024-01-16T14:59:42.798 INFO:tasks.ceph.osd.4.smithi204.stderr:2024-01-16T14:59:42.818+0000 7efcbaaa7500 -1 Falling back to public interface 2024-01-16T14:59:42.800 INFO:tasks.ceph.osd.5.smithi204.stderr:2024-01-16T14:59:42.819+0000 7f8c6c800640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6812/0: (98) Address already in use 2024-01-16T14:59:42.808 INFO:tasks.ceph.osd.3.smithi143.stderr:2024-01-16T14:59:42.827+0000 7fbbe7c1a500 -1 Falling back to public interface 2024-01-16T14:59:42.820 INFO:tasks.ceph.osd.7.smithi204.stderr:2024-01-16T14:59:42.839+0000 7f554b548500 -1 Falling back to public interface 2024-01-16T14:59:43.107 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T14:59:43.125+0000 7f27f9844040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-16T14:59:43.156 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-16T14:59:43.176+0000 7feee7830040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-16T14:59:43.251 DEBUG:teuthology.orchestra.run.smithi143:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-01-16T14:59:43.412 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.431+0000 7fac04000640 1 Processor -- start 2024-01-16T14:59:43.413 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.431+0000 7fac04000640 1 -- start start 2024-01-16T14:59:43.414 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.432+0000 7fac04000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabfc15d450 0x7fabfc0ce070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:43.414 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.432+0000 7fac04000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fabfc0ce5b0 0x7fabfc0b7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:43.414 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.432+0000 7fac04000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabfc0b7e20 con 0x7fabfc0b9070 2024-01-16T14:59:43.414 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.432+0000 7fac04000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabfc0b8100 con 0x7fabfc0ce5b0 2024-01-16T14:59:43.414 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.432+0000 7fac04000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabfc0b83e0 con 0x7fabfc15d450 2024-01-16T14:59:43.414 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.433+0000 7fac02000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fabfc0ce5b0 0x7fabfc0b7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:43.415 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.433+0000 7fac00c00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fabfc0b9070 0x7fabfc0d1900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51662/0 (socket says 172.21.15.143:51662) 2024-01-16T14:59:43.415 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.433+0000 7fac00c00640 1 -- 172.21.15.143:0/2877772462 learned_addr learned my addr 172.21.15.143:0/2877772462 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T14:59:43.415 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.433+0000 7fac01600640 1 --2- 172.21.15.143:0/2877772462 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabfc15d450 0x7fabfc0ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:43.415 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.433+0000 7fac02000640 1 -- 172.21.15.143:0/2877772462 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabfc15d450 msgr2=0x7fabfc0ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:59:43.415 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.433+0000 7fac02000640 1 --2- 172.21.15.143:0/2877772462 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabfc15d450 0x7fabfc0ce070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:43.415 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.433+0000 7fac02000640 1 -- 172.21.15.143:0/2877772462 >> v1:172.21.15.143:6789/0 conn(0x7fabfc0b9070 legacy=0x7fabfc0d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:43.415 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.433+0000 7fac02000640 1 -- 172.21.15.143:0/2877772462 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabfc0b86c0 con 0x7fabfc0ce5b0 2024-01-16T14:59:43.416 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.434+0000 7fac02000640 1 --2- 172.21.15.143:0/2877772462 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabfc0ce5b0 0x7fabfc0b7420 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fabf4009d90 tx=0x7fabf402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=127c6a3f82558978 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:43.416 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.434+0000 7fabfbe00640 1 -- 172.21.15.143:0/2877772462 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf402cc60 con 0x7fabfc0ce5b0 2024-01-16T14:59:43.416 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.434+0000 7fabfbe00640 1 -- 172.21.15.143:0/2877772462 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf402cdc0 con 0x7fabfc0ce5b0 2024-01-16T14:59:43.416 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.434+0000 7fabfbe00640 1 -- 172.21.15.143:0/2877772462 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf4034660 con 0x7fabfc0ce5b0 2024-01-16T14:59:43.417 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.435+0000 7fac04000640 1 -- 172.21.15.143:0/2877772462 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabfc0ce5b0 msgr2=0x7fabfc0b7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:43.417 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.435+0000 7fac04000640 1 --2- 172.21.15.143:0/2877772462 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabfc0ce5b0 0x7fabfc0b7420 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fabf4009d90 tx=0x7fabf402beb0 comp rx=0 tx=0).stop 2024-01-16T14:59:43.417 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.435+0000 7fac04000640 1 -- 172.21.15.143:0/2877772462 shutdown_connections 2024-01-16T14:59:43.417 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.435+0000 7fac04000640 1 --2- 172.21.15.143:0/2877772462 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabfc0ce5b0 0x7fabfc0b7420 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:43.417 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.435+0000 7fac04000640 1 --2- 172.21.15.143:0/2877772462 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabfc15d450 0x7fabfc0ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:43.417 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.435+0000 7fac04000640 1 -- 172.21.15.143:0/2877772462 >> 172.21.15.143:0/2877772462 conn(0x7fabfc071fb0 msgr2=0x7fabfc0723c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:59:43.417 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.435+0000 7fac04000640 1 -- 172.21.15.143:0/2877772462 shutdown_connections 2024-01-16T14:59:43.417 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.435+0000 7fac04000640 1 -- 172.21.15.143:0/2877772462 wait complete. 2024-01-16T14:59:43.418 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.437+0000 7fac04000640 1 Processor -- start 2024-01-16T14:59:43.418 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.437+0000 7fac04000640 1 -- start start 2024-01-16T14:59:43.420 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.438+0000 7fac04000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabfc0b9070 0x7fabfc0ca470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:43.420 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.438+0000 7fac04000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabfc0ce5b0 0x7fabfc14d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:43.421 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.438+0000 7fac04000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabfc15d450 0x7fabfc14de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:43.421 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.438+0000 7fac04000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabfc0c9600 con 0x7fabfc0b9070 2024-01-16T14:59:43.421 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.438+0000 7fac04000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabfc15ea60 con 0x7fabfc15d450 2024-01-16T14:59:43.421 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.438+0000 7fac04000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabfc15ed40 con 0x7fabfc0ce5b0 2024-01-16T14:59:43.421 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.438+0000 7fac01600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabfc0b9070 0x7fabfc0ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:43.421 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.438+0000 7fac01600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabfc0b9070 0x7fabfc0ca470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52698/0 (socket says 172.21.15.143:52698) 2024-01-16T14:59:43.421 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.438+0000 7fac01600640 1 -- 172.21.15.143:0/566546688 learned_addr learned my addr 172.21.15.143:0/566546688 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T14:59:43.421 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.438+0000 7fac02000640 1 --2- 172.21.15.143:0/566546688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabfc15d450 0x7fabfc14de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:43.421 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.438+0000 7fac00c00640 1 --2- 172.21.15.143:0/566546688 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabfc0ce5b0 0x7fabfc14d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:43.421 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.438+0000 7fac01600640 1 -- 172.21.15.143:0/566546688 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabfc0ce5b0 msgr2=0x7fabfc14d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:43.421 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.438+0000 7fac01600640 1 --2- 172.21.15.143:0/566546688 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabfc0ce5b0 0x7fabfc14d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:43.421 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.438+0000 7fac01600640 1 -- 172.21.15.143:0/566546688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabfc15d450 msgr2=0x7fabfc14de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:43.421 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.438+0000 7fac01600640 1 --2- 172.21.15.143:0/566546688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabfc15d450 0x7fabfc14de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:43.422 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.438+0000 7fac01600640 1 -- 172.21.15.143:0/566546688 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf4009a70 con 0x7fabfc0b9070 2024-01-16T14:59:43.422 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.438+0000 7fac02000640 1 --2- 172.21.15.143:0/566546688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabfc15d450 0x7fabfc14de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:59:43.422 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.439+0000 7fac01600640 1 --2- 172.21.15.143:0/566546688 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabfc0b9070 0x7fabfc0ca470 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fabec00d5f0 tx=0x7fabec00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:43.422 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.439+0000 7fabf9600640 1 -- 172.21.15.143:0/566546688 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabec0125f0 con 0x7fabfc0b9070 2024-01-16T14:59:43.422 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.439+0000 7fabf9600640 1 -- 172.21.15.143:0/566546688 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabec012750 con 0x7fabfc0b9070 2024-01-16T14:59:43.422 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.439+0000 7fabf9600640 1 -- 172.21.15.143:0/566546688 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabec012b00 con 0x7fabfc0b9070 2024-01-16T14:59:43.422 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.439+0000 7fac04000640 1 -- 172.21.15.143:0/566546688 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabfc15f020 con 0x7fabfc0b9070 2024-01-16T14:59:43.422 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.439+0000 7fac04000640 1 -- 172.21.15.143:0/566546688 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabfc14eaf0 con 0x7fabfc0b9070 2024-01-16T14:59:43.422 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.439+0000 7fabf9600640 1 -- 172.21.15.143:0/566546688 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fabec01e460 con 0x7fabfc0b9070 2024-01-16T14:59:43.422 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.439+0000 7fabf9600640 1 -- 172.21.15.143:0/566546688 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fabec0144d0 con 0x7fabfc0b9070 2024-01-16T14:59:43.423 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.440+0000 7fac04000640 1 -- 172.21.15.143:0/566546688 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabfc067990 con 0x7fabfc0b9070 2024-01-16T14:59:43.424 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.443+0000 7fabf9600640 1 -- 172.21.15.143:0/566546688 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fabec01e820 con 0x7fabfc0b9070 2024-01-16T14:59:43.435 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T14:59:43.454+0000 7f27f9844040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-16T14:59:43.460 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-16T14:59:43.480+0000 7feee7830040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-16T14:59:43.490 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.509+0000 7fac04000640 1 -- 172.21.15.143:0/566546688 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fabfc15e8c0 con 0x7fabfc0b9070 2024-01-16T14:59:43.493 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.512+0000 7fabf9600640 1 -- 172.21.15.143:0/566546688 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fabfc15e8c0 con 0x7fabfc0b9070 2024-01-16T14:59:43.493 INFO:teuthology.misc.health.smithi143.stdout: 2024-01-16T14:59:43.493 INFO:teuthology.misc.health.smithi143.stdout:{"epoch":10,"fsid":"c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0","created":"2024-01-16T14:59:38.532601+0000","modified":"2024-01-16T14:59:42.056297+0000","last_up_change":"0.000000","last_in_change":"2024-01-16T14:59:42.056297+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f4c8891d-ab29-472a-907d-8446f20f7012","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c4a2788d-8b89-4992-82b2-7f4b9debd509","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"64645c1d-31b2-4794-b0f2-7ccf8df30393","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"73674129-a84e-4d31-8247-7d0179ae6df2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"cea28c9b-2f1c-4087-b415-9960789131d6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"33b63560-5574-4e78-983f-b398d67a2cb6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"df397427-8940-4eee-9c79-eece669d735d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a2e5a15c-8312-405a-ac02-f7e1fc9df2e4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T14:59:43.495 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.513+0000 7fac04000640 1 -- 172.21.15.143:0/566546688 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabfc0b9070 msgr2=0x7fabfc0ca470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:43.495 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.513+0000 7fac04000640 1 --2- 172.21.15.143:0/566546688 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabfc0b9070 0x7fabfc0ca470 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fabec00d5f0 tx=0x7fabec00dab0 comp rx=0 tx=0).stop 2024-01-16T14:59:43.495 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.513+0000 7fac04000640 1 -- 172.21.15.143:0/566546688 shutdown_connections 2024-01-16T14:59:43.495 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.513+0000 7fac04000640 1 --2- 172.21.15.143:0/566546688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabfc15d450 0x7fabfc14de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:43.495 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.513+0000 7fac04000640 1 --2- 172.21.15.143:0/566546688 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabfc0ce5b0 0x7fabfc14d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:43.495 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.513+0000 7fac04000640 1 --2- 172.21.15.143:0/566546688 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabfc0b9070 0x7fabfc0ca470 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:43.495 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.513+0000 7fac04000640 1 -- 172.21.15.143:0/566546688 >> 172.21.15.143:0/566546688 conn(0x7fabfc071fb0 msgr2=0x7fabfc0b6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:59:43.495 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.513+0000 7fac04000640 1 -- 172.21.15.143:0/566546688 shutdown_connections 2024-01-16T14:59:43.495 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:43.513+0000 7fac04000640 1 -- 172.21.15.143:0/566546688 wait complete. 2024-01-16T14:59:43.500 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T14:59:43.518+0000 7f27f9844040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-16T14:59:43.507 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-01-16T14:59:43.521 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-16T14:59:43.540+0000 7feee7830040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-16T14:59:43.613 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T14:59:43.632+0000 7f27f9844040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-16T14:59:43.635 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-16T14:59:43.655+0000 7feee7830040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-16T14:59:44.651 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T14:59:44.670+0000 7f279a800640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-01-16T14:59:44.654 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T14:59:44.673+0000 7f279a800640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-01-16T14:59:46.312 INFO:tasks.ceph.osd.6.smithi204.stderr:2024-01-16T14:59:46.332+0000 7f0596e12500 -1 osd.6 0 log_to_monitors true 2024-01-16T14:59:46.340 INFO:tasks.ceph.osd.1.smithi143.stderr:2024-01-16T14:59:46.359+0000 7fed59994500 -1 osd.1 0 log_to_monitors true 2024-01-16T14:59:46.363 INFO:tasks.ceph.osd.4.smithi204.stderr:2024-01-16T14:59:46.383+0000 7efcbaaa7500 -1 osd.4 0 log_to_monitors true 2024-01-16T14:59:46.372 INFO:tasks.ceph.osd.5.smithi204.stderr:2024-01-16T14:59:46.392+0000 7f8c6dcfa500 -1 osd.5 0 log_to_monitors true 2024-01-16T14:59:46.387 INFO:tasks.ceph.osd.7.smithi204.stderr:2024-01-16T14:59:46.407+0000 7f554b548500 -1 osd.7 0 log_to_monitors true 2024-01-16T14:59:46.393 INFO:tasks.ceph.osd.3.smithi143.stderr:2024-01-16T14:59:46.412+0000 7fbbe7c1a500 -1 osd.3 0 log_to_monitors true 2024-01-16T14:59:46.596 INFO:tasks.ceph.osd.2.smithi143.stderr:2024-01-16T14:59:46.615+0000 7f5a9c202500 -1 osd.2 0 log_to_monitors true 2024-01-16T14:59:46.618 INFO:tasks.ceph.osd.0.smithi143.stderr:2024-01-16T14:59:46.637+0000 7fce92822500 -1 osd.0 0 log_to_monitors true 2024-01-16T14:59:49.964 INFO:tasks.ceph.osd.0.smithi143.stderr:2024-01-16T14:59:49.983+0000 7fce8c800640 -1 osd.0 0 waiting for initial osdmap 2024-01-16T14:59:49.975 INFO:tasks.ceph.osd.0.smithi143.stderr:2024-01-16T14:59:49.994+0000 7fce83200640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-16T14:59:49.989 INFO:tasks.ceph.osd.3.smithi143.stderr:2024-01-16T14:59:50.008+0000 7fbbe1c00640 -1 osd.3 0 waiting for initial osdmap 2024-01-16T14:59:49.996 INFO:tasks.ceph.osd.3.smithi143.stderr:2024-01-16T14:59:50.015+0000 7fbbd8600640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-16T14:59:50.043 INFO:tasks.ceph.osd.2.smithi143.stderr:2024-01-16T14:59:50.062+0000 7f5a96600640 -1 osd.2 0 waiting for initial osdmap 2024-01-16T14:59:50.055 INFO:tasks.ceph.osd.5.smithi204.stderr:2024-01-16T14:59:50.075+0000 7f8c6a000640 -1 osd.5 0 waiting for initial osdmap 2024-01-16T14:59:50.055 INFO:tasks.ceph.osd.2.smithi143.stderr:2024-01-16T14:59:50.074+0000 7f5a8d000640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-16T14:59:50.058 INFO:tasks.ceph.osd.4.smithi204.stderr:2024-01-16T14:59:50.074+0000 7efcb5000640 -1 osd.4 0 waiting for initial osdmap 2024-01-16T14:59:50.062 INFO:tasks.ceph.osd.1.smithi143.stderr:2024-01-16T14:59:50.081+0000 7fed53e00640 -1 osd.1 0 waiting for initial osdmap 2024-01-16T14:59:50.064 INFO:tasks.ceph.osd.6.smithi204.stderr:2024-01-16T14:59:50.084+0000 7f0590c00640 -1 osd.6 0 waiting for initial osdmap 2024-01-16T14:59:50.064 INFO:tasks.ceph.osd.5.smithi204.stderr:2024-01-16T14:59:50.084+0000 7f8c5ec00640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-16T14:59:50.070 INFO:tasks.ceph.osd.4.smithi204.stderr:2024-01-16T14:59:50.090+0000 7efcaba00640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-16T14:59:50.071 INFO:tasks.ceph.osd.1.smithi143.stderr:2024-01-16T14:59:50.090+0000 7fed4a800640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-16T14:59:50.073 INFO:tasks.ceph.osd.6.smithi204.stderr:2024-01-16T14:59:50.093+0000 7f0587600640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-16T14:59:50.078 INFO:tasks.ceph.osd.7.smithi204.stderr:2024-01-16T14:59:50.097+0000 7f5545a00640 -1 osd.7 0 waiting for initial osdmap 2024-01-16T14:59:50.089 INFO:tasks.ceph.osd.7.smithi204.stderr:2024-01-16T14:59:50.109+0000 7f553c400640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-16T14:59:50.312 DEBUG:teuthology.orchestra.run.smithi143:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-01-16T14:59:50.491 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.510+0000 7f6a01c00640 1 Processor -- start 2024-01-16T14:59:50.491 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.510+0000 7f6a01c00640 1 -- start start 2024-01-16T14:59:50.493 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.511+0000 7f6a01c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f69fc06c950 0x7f69fc071750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:50.493 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.512+0000 7f6a01c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69fc0b2a00 con 0x7f69fc0cc0c0 2024-01-16T14:59:50.493 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.512+0000 7f6a01c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69fc0b20d0 con 0x7f69fc1597c0 2024-01-16T14:59:50.493 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.512+0000 7f6a01c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69fc0cfb70 con 0x7f69fc06c950 2024-01-16T14:59:50.493 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.512+0000 7f69fb400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f69fc06c950 0x7f69fc071750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:50.493 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.512+0000 7f69f1e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f69fc0cc0c0 0x7f69fc0b2e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55456/0 (socket says 172.21.15.143:55456) 2024-01-16T14:59:50.493 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.512+0000 7f69f1e00640 1 -- 172.21.15.143:0/3514661747 learned_addr learned my addr 172.21.15.143:0/3514661747 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T14:59:50.494 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.512+0000 7f69fb400640 1 -- 172.21.15.143:0/3514661747 >> v1:172.21.15.204:6789/0 conn(0x7f69fc1597c0 legacy=0x7f69fc06c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:50.494 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.512+0000 7f69fb400640 1 -- 172.21.15.143:0/3514661747 >> v1:172.21.15.143:6789/0 conn(0x7f69fc0cc0c0 legacy=0x7f69fc0b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:50.494 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.512+0000 7f69fb400640 1 -- 172.21.15.143:0/3514661747 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69fc0b23b0 con 0x7f69fc06c950 2024-01-16T14:59:50.495 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.513+0000 7f69fb400640 1 --2- 172.21.15.143:0/3514661747 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69fc06c950 0x7f69fc071750 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f69f40097a0 tx=0x7f69f402ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c915384e7bf5d0f1 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:50.495 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.514+0000 7f69fa000640 1 -- 172.21.15.143:0/3514661747 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f402e830 con 0x7f69fc06c950 2024-01-16T14:59:50.495 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.514+0000 7f69fa000640 1 -- 172.21.15.143:0/3514661747 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f402e990 con 0x7f69fc06c950 2024-01-16T14:59:50.495 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.514+0000 7f69fa000640 1 -- 172.21.15.143:0/3514661747 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f402ed20 con 0x7f69fc06c950 2024-01-16T14:59:50.496 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.514+0000 7f6a01c00640 1 -- 172.21.15.143:0/3514661747 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69fc06c950 msgr2=0x7f69fc071750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:50.496 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.514+0000 7f6a01c00640 1 --2- 172.21.15.143:0/3514661747 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69fc06c950 0x7f69fc071750 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f69f40097a0 tx=0x7f69f402ca70 comp rx=0 tx=0).stop 2024-01-16T14:59:50.496 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.514+0000 7f6a01c00640 1 -- 172.21.15.143:0/3514661747 shutdown_connections 2024-01-16T14:59:50.496 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.514+0000 7f6a01c00640 1 --2- 172.21.15.143:0/3514661747 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69fc06c950 0x7f69fc071750 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:50.496 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.514+0000 7f6a01c00640 1 -- 172.21.15.143:0/3514661747 >> 172.21.15.143:0/3514661747 conn(0x7f69fc06a490 msgr2=0x7f69fc06a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:59:50.496 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.514+0000 7f6a01c00640 1 -- 172.21.15.143:0/3514661747 shutdown_connections 2024-01-16T14:59:50.496 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.514+0000 7f6a01c00640 1 -- 172.21.15.143:0/3514661747 wait complete. 2024-01-16T14:59:50.497 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.516+0000 7f6a01c00640 1 Processor -- start 2024-01-16T14:59:50.497 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.516+0000 7f6a01c00640 1 -- start start 2024-01-16T14:59:50.499 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.517+0000 7f6a01c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69fc06c950 0x7f69fc13b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:50.500 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.517+0000 7f6a01c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69fc0cc0c0 0x7f69fc14d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:50.500 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.517+0000 7f6a01c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69fc1597c0 0x7f69fc153c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:50.500 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.517+0000 7f6a01c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69fc14c110 con 0x7f69fc1597c0 2024-01-16T14:59:50.500 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.517+0000 7f6a01c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69fc13be70 con 0x7f69fc06c950 2024-01-16T14:59:50.500 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.517+0000 7f6a01c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69fc154ab0 con 0x7f69fc0cc0c0 2024-01-16T14:59:50.500 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.517+0000 7f69fb400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69fc1597c0 0x7f69fc153c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:50.500 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.517+0000 7f69fb400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69fc1597c0 0x7f69fc153c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44594/0 (socket says 172.21.15.143:44594) 2024-01-16T14:59:50.500 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.517+0000 7f69fb400640 1 -- 172.21.15.143:0/2997693634 learned_addr learned my addr 172.21.15.143:0/2997693634 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T14:59:50.500 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.517+0000 7f69faa00640 1 --2- 172.21.15.143:0/2997693634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69fc06c950 0x7f69fc13b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:50.500 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.517+0000 7f69fb400640 1 -- 172.21.15.143:0/2997693634 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69fc0cc0c0 msgr2=0x7f69fc14d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:50.500 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.517+0000 7f69fb400640 1 --2- 172.21.15.143:0/2997693634 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69fc0cc0c0 0x7f69fc14d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:50.500 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.517+0000 7f69fb400640 1 -- 172.21.15.143:0/2997693634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69fc06c950 msgr2=0x7f69fc13b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:50.500 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.517+0000 7f69fb400640 1 --2- 172.21.15.143:0/2997693634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69fc06c950 0x7f69fc13b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:50.501 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.517+0000 7f69fb400640 1 -- 172.21.15.143:0/2997693634 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f4009400 con 0x7f69fc1597c0 2024-01-16T14:59:50.501 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.517+0000 7f69faa00640 1 --2- 172.21.15.143:0/2997693634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69fc06c950 0x7f69fc13b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:59:50.501 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.518+0000 7f69fb400640 1 --2- 172.21.15.143:0/2997693634 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69fc1597c0 0x7f69fc153c60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f69f40098c0 tx=0x7f69f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:50.501 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.518+0000 7f69f3400640 1 -- 172.21.15.143:0/2997693634 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f403d070 con 0x7f69fc1597c0 2024-01-16T14:59:50.501 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.518+0000 7f69f3400640 1 -- 172.21.15.143:0/2997693634 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f402e7d0 con 0x7f69fc1597c0 2024-01-16T14:59:50.501 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.518+0000 7f69f3400640 1 -- 172.21.15.143:0/2997693634 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f4038e80 con 0x7f69fc1597c0 2024-01-16T14:59:50.501 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.518+0000 7f6a01c00640 1 -- 172.21.15.143:0/2997693634 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69fc14b490 con 0x7f69fc1597c0 2024-01-16T14:59:50.501 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.518+0000 7f6a01c00640 1 -- 172.21.15.143:0/2997693634 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69fc14ba50 con 0x7f69fc1597c0 2024-01-16T14:59:50.501 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.519+0000 7f6a01c00640 1 -- 172.21.15.143:0/2997693634 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69fc067990 con 0x7f69fc1597c0 2024-01-16T14:59:50.502 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.520+0000 7f69f3400640 1 -- 172.21.15.143:0/2997693634 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 4) v1 ==== 89483+0+0 (secure 0 0 0) 0x7f69f404a400 con 0x7f69fc1597c0 2024-01-16T14:59:50.502 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.521+0000 7f69f3400640 1 --2- 172.21.15.143:0/2997693634 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f69d406b230 0x7f69d406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:50.502 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.521+0000 7f69f3400640 1 -- 172.21.15.143:0/2997693634 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f69f40aab70 con 0x7f69fc1597c0 2024-01-16T14:59:50.502 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.521+0000 7f69faa00640 1 --2- 172.21.15.143:0/2997693634 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f69d406b230 0x7f69d406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:50.503 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.522+0000 7f69faa00640 1 --2- 172.21.15.143:0/2997693634 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f69d406b230 0x7f69d406d6f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f69fc14d390 tx=0x7f69ec009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:50.507 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.526+0000 7f69f3400640 1 -- 172.21.15.143:0/2997693634 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f69f40768e0 con 0x7f69fc1597c0 2024-01-16T14:59:50.687 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.705+0000 7f6a01c00640 1 -- 172.21.15.143:0/2997693634 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69fc0cb250 con 0x7f69fc1597c0 2024-01-16T14:59:50.688 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.707+0000 7f69f3400640 1 -- 172.21.15.143:0/2997693634 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f69f4079f00 con 0x7f69fc1597c0 2024-01-16T14:59:50.688 INFO:teuthology.misc.health.smithi143.stdout: 2024-01-16T14:59:50.688 INFO:teuthology.misc.health.smithi143.stdout:{"epoch":12,"fsid":"c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0","created":"2024-01-16T14:59:38.532601+0000","modified":"2024-01-16T14:59:47.664508+0000","last_up_change":"0.000000","last_in_change":"2024-01-16T14:59:42.056297+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f4c8891d-ab29-472a-907d-8446f20f7012","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c4a2788d-8b89-4992-82b2-7f4b9debd509","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"64645c1d-31b2-4794-b0f2-7ccf8df30393","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"73674129-a84e-4d31-8247-7d0179ae6df2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"cea28c9b-2f1c-4087-b415-9960789131d6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"33b63560-5574-4e78-983f-b398d67a2cb6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"df397427-8940-4eee-9c79-eece669d735d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a2e5a15c-8312-405a-ac02-f7e1fc9df2e4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T14:59:50.691 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.709+0000 7f6a01c00640 1 -- 172.21.15.143:0/2997693634 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f69d406b230 msgr2=0x7f69d406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:50.691 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.709+0000 7f6a01c00640 1 --2- 172.21.15.143:0/2997693634 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f69d406b230 0x7f69d406d6f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f69fc14d390 tx=0x7f69ec009040 comp rx=0 tx=0).stop 2024-01-16T14:59:50.691 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.709+0000 7f6a01c00640 1 -- 172.21.15.143:0/2997693634 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69fc1597c0 msgr2=0x7f69fc153c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:50.691 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.709+0000 7f6a01c00640 1 --2- 172.21.15.143:0/2997693634 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69fc1597c0 0x7f69fc153c60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f69f40098c0 tx=0x7f69f40046c0 comp rx=0 tx=0).stop 2024-01-16T14:59:50.691 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.709+0000 7f6a01c00640 1 -- 172.21.15.143:0/2997693634 shutdown_connections 2024-01-16T14:59:50.691 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.709+0000 7f6a01c00640 1 --2- 172.21.15.143:0/2997693634 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f69d406b230 0x7f69d406d6f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:50.691 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.709+0000 7f6a01c00640 1 --2- 172.21.15.143:0/2997693634 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69fc1597c0 0x7f69fc153c60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:50.691 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.709+0000 7f6a01c00640 1 --2- 172.21.15.143:0/2997693634 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69fc0cc0c0 0x7f69fc14d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:50.691 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.709+0000 7f6a01c00640 1 --2- 172.21.15.143:0/2997693634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69fc06c950 0x7f69fc13b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:50.691 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.709+0000 7f6a01c00640 1 -- 172.21.15.143:0/2997693634 >> 172.21.15.143:0/2997693634 conn(0x7f69fc06a490 msgr2=0x7f69fc1517f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:59:50.691 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.709+0000 7f6a01c00640 1 -- 172.21.15.143:0/2997693634 shutdown_connections 2024-01-16T14:59:50.692 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:50.710+0000 7f6a01c00640 1 -- 172.21.15.143:0/2997693634 wait complete. 2024-01-16T14:59:50.703 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-01-16T14:59:52.847 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T14:59:52.858 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T14:59:57.509 DEBUG:teuthology.orchestra.run.smithi143:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-01-16T14:59:57.691 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.710+0000 7ff399e00640 1 Processor -- start 2024-01-16T14:59:57.691 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.710+0000 7ff399e00640 1 -- start start 2024-01-16T14:59:57.692 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.711+0000 7ff399e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff39406b5a0 0x7ff39406ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:57.692 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.711+0000 7ff399e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3940cb950 con 0x7ff394157ff0 2024-01-16T14:59:57.692 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.711+0000 7ff399e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff39406c110 con 0x7ff3940b2cc0 2024-01-16T14:59:57.693 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.711+0000 7ff399e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3940cbbf0 con 0x7ff39406b5a0 2024-01-16T14:59:57.693 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.711+0000 7ff392a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff394157ff0 0x7ff3940cc6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55002/0 (socket says 172.21.15.143:55002) 2024-01-16T14:59:57.693 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.711+0000 7ff392a00640 1 -- 172.21.15.143:0/2906776819 learned_addr learned my addr 172.21.15.143:0/2906776819 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T14:59:57.693 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.711+0000 7ff38be00640 1 --2- 172.21.15.143:0/2906776819 >> v2:172.21.15.143:3300/0 conn(0x7ff39406b5a0 0x7ff39406ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:57.693 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.712+0000 7ff38be00640 1 -- 172.21.15.143:0/2906776819 >> v1:172.21.15.204:6789/0 conn(0x7ff3940b2cc0 legacy=0x7ff39415efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:59:57.693 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.712+0000 7ff392000640 1 -- 172.21.15.143:0/2906776819 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2217083270 0 0) 0x7ff3940cb950 con 0x7ff394157ff0 2024-01-16T14:59:57.694 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.712+0000 7ff38be00640 1 -- 172.21.15.143:0/2906776819 >> v1:172.21.15.143:6789/0 conn(0x7ff394157ff0 legacy=0x7ff3940cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:57.694 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.712+0000 7ff38be00640 1 -- 172.21.15.143:0/2906776819 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3940cbed0 con 0x7ff39406b5a0 2024-01-16T14:59:57.694 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.713+0000 7ff38be00640 1 --2- 172.21.15.143:0/2906776819 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff39406b5a0 0x7ff39406ba20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff37c0097a0 tx=0x7ff37c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ea821d2c5f1c8c server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:57.694 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.713+0000 7ff392000640 1 -- 172.21.15.143:0/2906776819 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c031830 con 0x7ff39406b5a0 2024-01-16T14:59:57.694 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.713+0000 7ff392000640 1 -- 172.21.15.143:0/2906776819 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff37c031990 con 0x7ff39406b5a0 2024-01-16T14:59:57.695 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.713+0000 7ff392000640 1 -- 172.21.15.143:0/2906776819 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c031d20 con 0x7ff39406b5a0 2024-01-16T14:59:57.695 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.714+0000 7ff399e00640 1 -- 172.21.15.143:0/2906776819 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff39406b5a0 msgr2=0x7ff39406ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:57.695 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.714+0000 7ff399e00640 1 --2- 172.21.15.143:0/2906776819 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff39406b5a0 0x7ff39406ba20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff37c0097a0 tx=0x7ff37c02b8d0 comp rx=0 tx=0).stop 2024-01-16T14:59:57.695 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.714+0000 7ff399e00640 1 -- 172.21.15.143:0/2906776819 shutdown_connections 2024-01-16T14:59:57.696 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.714+0000 7ff399e00640 1 --2- 172.21.15.143:0/2906776819 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff39406b5a0 0x7ff39406ba20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:57.696 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.714+0000 7ff399e00640 1 -- 172.21.15.143:0/2906776819 >> 172.21.15.143:0/2906776819 conn(0x7ff394070110 msgr2=0x7ff3940704f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:59:57.696 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.714+0000 7ff399e00640 1 -- 172.21.15.143:0/2906776819 shutdown_connections 2024-01-16T14:59:57.696 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.714+0000 7ff399e00640 1 -- 172.21.15.143:0/2906776819 wait complete. 2024-01-16T14:59:57.697 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.716+0000 7ff399e00640 1 Processor -- start 2024-01-16T14:59:57.697 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.716+0000 7ff399e00640 1 -- start start 2024-01-16T14:59:57.699 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.717+0000 7ff399e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3940b2cc0 0x7ff3940c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:57.699 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.717+0000 7ff399e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff394157ff0 0x7ff394155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:57.699 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.717+0000 7ff399e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3941f2e20 0x7ff3940b5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:57.699 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.717+0000 7ff399e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff394155b70 con 0x7ff3940b2cc0 2024-01-16T14:59:57.699 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.717+0000 7ff399e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3941563a0 con 0x7ff3941f2e20 2024-01-16T14:59:57.699 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.717+0000 7ff399e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff394155de0 con 0x7ff394157ff0 2024-01-16T14:59:57.699 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.717+0000 7ff392a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3940b2cc0 0x7ff3940c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:57.699 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.717+0000 7ff392a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3940b2cc0 0x7ff3940c7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56950/0 (socket says 172.21.15.143:56950) 2024-01-16T14:59:57.699 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.717+0000 7ff392a00640 1 -- 172.21.15.143:0/372849795 learned_addr learned my addr 172.21.15.143:0/372849795 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T14:59:57.699 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.717+0000 7ff392a00640 1 -- 172.21.15.143:0/372849795 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff394157ff0 msgr2=0x7ff394155630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:57.699 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.717+0000 7ff392a00640 1 --2- 172.21.15.143:0/372849795 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff394157ff0 0x7ff394155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:57.699 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.717+0000 7ff392a00640 1 -- 172.21.15.143:0/372849795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3941f2e20 msgr2=0x7ff3940b5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:57.699 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.717+0000 7ff392a00640 1 --2- 172.21.15.143:0/372849795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3941f2e20 0x7ff3940b5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:57.700 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.717+0000 7ff392a00640 1 -- 172.21.15.143:0/372849795 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff37c009400 con 0x7ff3940b2cc0 2024-01-16T14:59:57.700 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.718+0000 7ff392a00640 1 --2- 172.21.15.143:0/372849795 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3940b2cc0 0x7ff3940c7b90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff38400c430 tx=0x7ff38400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:57.700 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.718+0000 7ff38b400640 1 -- 172.21.15.143:0/372849795 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3840074e0 con 0x7ff3940b2cc0 2024-01-16T14:59:57.701 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.718+0000 7ff38b400640 1 -- 172.21.15.143:0/372849795 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff384007660 con 0x7ff3940b2cc0 2024-01-16T14:59:57.701 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.718+0000 7ff38b400640 1 -- 172.21.15.143:0/372849795 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff384004a20 con 0x7ff3940b2cc0 2024-01-16T14:59:57.701 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.718+0000 7ff399e00640 1 -- 172.21.15.143:0/372849795 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3941560c0 con 0x7ff3940b2cc0 2024-01-16T14:59:57.701 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.718+0000 7ff399e00640 1 -- 172.21.15.143:0/372849795 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3940c51c0 con 0x7ff3940b2cc0 2024-01-16T14:59:57.701 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.719+0000 7ff399e00640 1 -- 172.21.15.143:0/372849795 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3940c53b0 con 0x7ff3940b2cc0 2024-01-16T14:59:57.701 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.720+0000 7ff38b400640 1 -- 172.21.15.143:0/372849795 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7ff384004b80 con 0x7ff3940b2cc0 2024-01-16T14:59:57.702 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.721+0000 7ff38b400640 1 --2- 172.21.15.143:0/372849795 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7ff36c06b2c0 0x7ff36c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:57.703 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.722+0000 7ff38b400640 1 -- 172.21.15.143:0/372849795 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff3840896c0 con 0x7ff3940b2cc0 2024-01-16T14:59:57.703 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.722+0000 7ff38be00640 1 --2- 172.21.15.143:0/372849795 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7ff36c06b2c0 0x7ff36c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:57.703 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.722+0000 7ff38be00640 1 --2- 172.21.15.143:0/372849795 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7ff36c06b2c0 0x7ff36c06d780 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff37c036000 tx=0x7ff37c030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:57.708 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.727+0000 7ff38b400640 1 -- 172.21.15.143:0/372849795 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff384054870 con 0x7ff3940b2cc0 2024-01-16T14:59:57.897 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.915+0000 7ff399e00640 1 -- 172.21.15.143:0/372849795 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3940748f0 con 0x7ff3940b2cc0 2024-01-16T14:59:57.899 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.918+0000 7ff38b400640 1 -- 172.21.15.143:0/372849795 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12910 (secure 0 0 0) 0x7ff384057e90 con 0x7ff3940b2cc0 2024-01-16T14:59:57.899 INFO:teuthology.misc.health.smithi143.stdout: 2024-01-16T14:59:57.899 INFO:teuthology.misc.health.smithi143.stdout:{"epoch":16,"fsid":"c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0","created":"2024-01-16T14:59:38.532601+0000","modified":"2024-01-16T14:59:54.038678+0000","last_up_change":"2024-01-16T14:59:50.983699+0000","last_in_change":"2024-01-16T14:59:42.056297+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:59:51.727720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f4c8891d-ab29-472a-907d-8446f20f7012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6801","nonce":1569293626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6805","nonce":1569293626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6814","nonce":1569293626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6809","nonce":1569293626}]},"public_addr":"172.21.15.143:6801/1569293626","cluster_addr":"172.21.15.143:6805/1569293626","heartbeat_back_addr":"172.21.15.143:6814/1569293626","heartbeat_front_addr":"172.21.15.143:6809/1569293626","state":["exists","up"]},{"osd":1,"uuid":"c4a2788d-8b89-4992-82b2-7f4b9debd509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6804","nonce":1276841168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6808","nonce":1276841168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6813","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6815","nonce":1276841168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6811","nonce":1276841168}]},"public_addr":"172.21.15.143:6804/1276841168","cluster_addr":"172.21.15.143:6808/1276841168","heartbeat_back_addr":"172.21.15.143:6815/1276841168","heartbeat_front_addr":"172.21.15.143:6811/1276841168","state":["exists","up"]},{"osd":2,"uuid":"64645c1d-31b2-4794-b0f2-7ccf8df30393","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6817","nonce":1217670170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6819","nonce":1217670170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6823","nonce":1217670170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6821","nonce":1217670170}]},"public_addr":"172.21.15.143:6817/1217670170","cluster_addr":"172.21.15.143:6819/1217670170","heartbeat_back_addr":"172.21.15.143:6823/1217670170","heartbeat_front_addr":"172.21.15.143:6821/1217670170","state":["exists","up"]},{"osd":3,"uuid":"73674129-a84e-4d31-8247-7d0179ae6df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6825","nonce":3414188012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6827","nonce":3414188012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6831","nonce":3414188012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6829","nonce":3414188012}]},"public_addr":"172.21.15.143:6825/3414188012","cluster_addr":"172.21.15.143:6827/3414188012","heartbeat_back_addr":"172.21.15.143:6831/3414188012","heartbeat_front_addr":"172.21.15.143:6829/3414188012","state":["exists","up"]},{"osd":4,"uuid":"cea28c9b-2f1c-4087-b415-9960789131d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6811","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6812","nonce":1070255716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6816","nonce":1070255716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6823","nonce":1070255716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6820","nonce":1070255716}]},"public_addr":"172.21.15.204:6812/1070255716","cluster_addr":"172.21.15.204:6816/1070255716","heartbeat_back_addr":"172.21.15.204:6823/1070255716","heartbeat_front_addr":"172.21.15.204:6820/1070255716","state":["exists","up"]},{"osd":5,"uuid":"33b63560-5574-4e78-983f-b398d67a2cb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6809","nonce":3605417837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6813","nonce":3605417837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6819","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6821","nonce":3605417837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6817","nonce":3605417837}]},"public_addr":"172.21.15.204:6809/3605417837","cluster_addr":"172.21.15.204:6813/3605417837","heartbeat_back_addr":"172.21.15.204:6821/3605417837","heartbeat_front_addr":"172.21.15.204:6817/3605417837","state":["exists","up"]},{"osd":6,"uuid":"df397427-8940-4eee-9c79-eece669d735d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6801","nonce":1085000534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6803","nonce":1085000534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6808","nonce":1085000534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6805","nonce":1085000534}]},"public_addr":"172.21.15.204:6801/1085000534","cluster_addr":"172.21.15.204:6803/1085000534","heartbeat_back_addr":"172.21.15.204:6808/1085000534","heartbeat_front_addr":"172.21.15.204:6805/1085000534","state":["exists","up"]},{"osd":7,"uuid":"a2e5a15c-8312-405a-ac02-f7e1fc9df2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6825","nonce":1337220609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6827","nonce":1337220609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6831","nonce":1337220609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6829","nonce":1337220609}]},"public_addr":"172.21.15.204:6825/1337220609","cluster_addr":"172.21.15.204:6827/1337220609","heartbeat_back_addr":"172.21.15.204:6831/1337220609","heartbeat_front_addr":"172.21.15.204:6829/1337220609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:49.984507+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.395306+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.632998+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.373627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.342920+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.438366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.331926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.406657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T14:59:57.902 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.920+0000 7ff399e00640 1 -- 172.21.15.143:0/372849795 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7ff36c06b2c0 msgr2=0x7ff36c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:57.902 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.920+0000 7ff399e00640 1 --2- 172.21.15.143:0/372849795 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7ff36c06b2c0 0x7ff36c06d780 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff37c036000 tx=0x7ff37c030000 comp rx=0 tx=0).stop 2024-01-16T14:59:57.902 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.921+0000 7ff399e00640 1 -- 172.21.15.143:0/372849795 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3940b2cc0 msgr2=0x7ff3940c7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:57.902 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.921+0000 7ff399e00640 1 --2- 172.21.15.143:0/372849795 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3940b2cc0 0x7ff3940c7b90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff38400c430 tx=0x7ff38400c8f0 comp rx=0 tx=0).stop 2024-01-16T14:59:57.903 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.921+0000 7ff399e00640 1 -- 172.21.15.143:0/372849795 shutdown_connections 2024-01-16T14:59:57.903 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.921+0000 7ff399e00640 1 --2- 172.21.15.143:0/372849795 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7ff36c06b2c0 0x7ff36c06d780 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:57.903 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.921+0000 7ff399e00640 1 --2- 172.21.15.143:0/372849795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3941f2e20 0x7ff3940b5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:57.903 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.921+0000 7ff399e00640 1 --2- 172.21.15.143:0/372849795 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff394157ff0 0x7ff394155630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:57.903 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.921+0000 7ff399e00640 1 --2- 172.21.15.143:0/372849795 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3940b2cc0 0x7ff3940c7b90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:57.903 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.921+0000 7ff399e00640 1 -- 172.21.15.143:0/372849795 >> 172.21.15.143:0/372849795 conn(0x7ff394070110 msgr2=0x7ff394066360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:59:57.903 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.921+0000 7ff399e00640 1 -- 172.21.15.143:0/372849795 shutdown_connections 2024-01-16T14:59:57.903 INFO:teuthology.misc.health.smithi143.stderr:2024-01-16T14:59:57.921+0000 7ff399e00640 1 -- 172.21.15.143:0/372849795 wait complete. 2024-01-16T14:59:57.916 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-01-16T14:59:57.916 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-01-16T14:59:57.916 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-16T14:59:57.916 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-16T14:59:57.923 INFO:tasks.ceph.mds.a:Started 2024-01-16T14:59:57.924 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-16T14:59:57.924 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-16T14:59:57.931 INFO:tasks.ceph.mds.c:Started 2024-01-16T14:59:57.931 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-16T14:59:57.931 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-16T14:59:57.939 INFO:tasks.ceph.mds.b:Started 2024-01-16T14:59:57.939 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-16T14:59:57.939 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-16T14:59:57.946 INFO:tasks.ceph.mds.d:Started 2024-01-16T14:59:57.946 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-01-16T14:59:57.947 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T14:59:58.077 INFO:tasks.ceph.mds.c.smithi143.stdout:starting mds.c at 2024-01-16T14:59:58.078 INFO:tasks.ceph.mds.a.smithi143.stdout:starting mds.a at 2024-01-16T14:59:58.104 INFO:tasks.ceph.mds.d.smithi204.stdout:starting mds.d at 2024-01-16T14:59:58.104 INFO:tasks.ceph.mds.b.smithi204.stdout:starting mds.b at 2024-01-16T14:59:58.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.149+0000 7f23a4c00640 1 Processor -- start 2024-01-16T14:59:58.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.149+0000 7f23a4c00640 1 -- start start 2024-01-16T14:59:58.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.151+0000 7f23a4c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f23a01461d0 0x7f23a00b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:58.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.151+0000 7f23a4c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f23a00b4fe0 0x7f23a00b3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:58.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.151+0000 7f23a4c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23a006d180 con 0x7f23a00b3890 2024-01-16T14:59:58.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.151+0000 7f23a4c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23a00b3d00 con 0x7f23a01461d0 2024-01-16T14:59:58.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.151+0000 7f23a4c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a006ba70 con 0x7f23a00b4fe0 2024-01-16T14:59:58.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.151+0000 7f2395200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f23a00b4fe0 0x7f23a00b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:58.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.151+0000 7f2395200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f23a00b4fe0 0x7f23a00b3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:43076/0 (socket says 172.21.15.143:43076) 2024-01-16T14:59:58.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.151+0000 7f239de00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f23a01461d0 0x7f23a00b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:58.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.151+0000 7f2395200640 1 -- 172.21.15.143:0/2526327498 learned_addr learned my addr 172.21.15.143:0/2526327498 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T14:59:58.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.151+0000 7f239e800640 1 --1- 172.21.15.143:0/2526327498 >> v1:172.21.15.204:6789/0 conn(0x7f23a00b3890 0x7f23a015dae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:44924/0 (socket says 172.21.15.143:44924) 2024-01-16T14:59:58.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.152+0000 7f2395200640 1 -- 172.21.15.143:0/2526327498 >> v2:172.21.15.204:3300/0 conn(0x7f23a01461d0 msgr2=0x7f23a00b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:59:58.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.152+0000 7f2395200640 1 --2- 172.21.15.143:0/2526327498 >> v2:172.21.15.204:3300/0 conn(0x7f23a01461d0 0x7f23a00b4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:58.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.152+0000 7f2395200640 1 -- 172.21.15.143:0/2526327498 >> v1:172.21.15.204:6789/0 conn(0x7f23a00b3890 legacy=0x7f23a015dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:58.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.152+0000 7f2395200640 1 -- 172.21.15.143:0/2526327498 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a006bd50 con 0x7f23a00b4fe0 2024-01-16T14:59:58.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.153+0000 7f2395200640 1 --2- 172.21.15.143:0/2526327498 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f23a00b4fe0 0x7f23a00b3350 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2388009870 tx=0x7f238802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=999f31585e46f634 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:58.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.153+0000 7f239d400640 1 -- 172.21.15.143:0/2526327498 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2388030840 con 0x7f23a00b4fe0 2024-01-16T14:59:58.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.153+0000 7f239d400640 1 -- 172.21.15.143:0/2526327498 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23880309a0 con 0x7f23a00b4fe0 2024-01-16T14:59:58.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.153+0000 7f239d400640 1 -- 172.21.15.143:0/2526327498 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2388030d30 con 0x7f23a00b4fe0 2024-01-16T14:59:58.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.153+0000 7f23a4c00640 1 -- 172.21.15.143:0/2526327498 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f23a00b4fe0 msgr2=0x7f23a00b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:58.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.153+0000 7f23a4c00640 1 --2- 172.21.15.143:0/2526327498 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f23a00b4fe0 0x7f23a00b3350 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2388009870 tx=0x7f238802b980 comp rx=0 tx=0).stop 2024-01-16T14:59:58.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.154+0000 7f23a4c00640 1 -- 172.21.15.143:0/2526327498 shutdown_connections 2024-01-16T14:59:58.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.154+0000 7f23a4c00640 1 --2- 172.21.15.143:0/2526327498 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f23a00b4fe0 0x7f23a00b3350 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:58.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.154+0000 7f23a4c00640 1 --2- 172.21.15.143:0/2526327498 >> v2:172.21.15.204:3300/0 conn(0x7f23a01461d0 0x7f23a00b4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:58.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.154+0000 7f23a4c00640 1 -- 172.21.15.143:0/2526327498 >> 172.21.15.143:0/2526327498 conn(0x7f23a0061340 msgr2=0x7f23a0061750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:59:58.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.154+0000 7f23a4c00640 1 -- 172.21.15.143:0/2526327498 shutdown_connections 2024-01-16T14:59:58.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.154+0000 7f23a4c00640 1 -- 172.21.15.143:0/2526327498 wait complete. 2024-01-16T14:59:58.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.155+0000 7f23a4c00640 1 Processor -- start 2024-01-16T14:59:58.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.156+0000 7f23a4c00640 1 -- start start 2024-01-16T14:59:58.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.156+0000 7f23a4c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23a00b3890 0x7f23a015c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:58.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.156+0000 7f23a4c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f23a00b4fe0 0x7f23a00cdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:58.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.156+0000 7f23a4c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23a01461d0 0x7f23a00a9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:58.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.156+0000 7f23a4c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a00ce8a0 con 0x7f23a00b3890 2024-01-16T14:59:58.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.156+0000 7f23a4c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a00ceb80 con 0x7f23a01461d0 2024-01-16T14:59:58.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.156+0000 7f23a4c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a00cee60 con 0x7f23a00b4fe0 2024-01-16T14:59:58.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.157+0000 7f239de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23a00b3890 0x7f23a015c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:58.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.157+0000 7f239de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23a00b3890 0x7f23a015c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56986/0 (socket says 172.21.15.143:56986) 2024-01-16T14:59:58.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.157+0000 7f239de00640 1 -- 172.21.15.143:0/3356152361 learned_addr learned my addr 172.21.15.143:0/3356152361 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T14:59:58.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.157+0000 7f239e800640 1 --2- 172.21.15.143:0/3356152361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23a01461d0 0x7f23a00a9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:58.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.157+0000 7f2395200640 1 --2- 172.21.15.143:0/3356152361 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f23a00b4fe0 0x7f23a00cdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:58.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.157+0000 7f239de00640 1 -- 172.21.15.143:0/3356152361 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f23a00b4fe0 msgr2=0x7f23a00cdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:58.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.157+0000 7f239de00640 1 --2- 172.21.15.143:0/3356152361 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f23a00b4fe0 0x7f23a00cdfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:58.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.157+0000 7f239de00640 1 -- 172.21.15.143:0/3356152361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23a01461d0 msgr2=0x7f23a00a9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:58.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.157+0000 7f239de00640 1 --2- 172.21.15.143:0/3356152361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23a01461d0 0x7f23a00a9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:58.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.157+0000 7f239de00640 1 -- 172.21.15.143:0/3356152361 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23880094d0 con 0x7f23a00b3890 2024-01-16T14:59:58.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.157+0000 7f239e800640 1 --2- 172.21.15.143:0/3356152361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23a01461d0 0x7f23a00a9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:59:58.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.157+0000 7f239de00640 1 --2- 172.21.15.143:0/3356152361 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23a00b3890 0x7f23a015c220 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f239000e870 tx=0x7f239000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:58.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.158+0000 7f2396a00640 1 -- 172.21.15.143:0/3356152361 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23900094f0 con 0x7f23a00b3890 2024-01-16T14:59:58.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.158+0000 7f2396a00640 1 -- 172.21.15.143:0/3356152361 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f239000c940 con 0x7f23a00b3890 2024-01-16T14:59:58.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.158+0000 7f2396a00640 1 -- 172.21.15.143:0/3356152361 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239000ccf0 con 0x7f23a00b3890 2024-01-16T14:59:58.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.158+0000 7f23a4c00640 1 -- 172.21.15.143:0/3356152361 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23a00cfeb0 con 0x7f23a00b3890 2024-01-16T14:59:58.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.158+0000 7f23a4c00640 1 -- 172.21.15.143:0/3356152361 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23a015c760 con 0x7f23a00b3890 2024-01-16T14:59:58.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.159+0000 7f23a4c00640 1 -- 172.21.15.143:0/3356152361 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23a0067470 con 0x7f23a00b3890 2024-01-16T14:59:58.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.160+0000 7f2396a00640 1 -- 172.21.15.143:0/3356152361 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f2390013470 con 0x7f23a00b3890 2024-01-16T14:59:58.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.161+0000 7f2396a00640 1 --2- 172.21.15.143:0/3356152361 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f237806b2f0 0x7f237806d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:58.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.161+0000 7f2396a00640 1 -- 172.21.15.143:0/3356152361 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f23900899d0 con 0x7f23a00b3890 2024-01-16T14:59:58.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.161+0000 7f2395200640 1 --2- 172.21.15.143:0/3356152361 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f237806b2f0 0x7f237806d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:58.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.162+0000 7f2395200640 1 --2- 172.21.15.143:0/3356152361 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f237806b2f0 0x7f237806d7b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2388036000 tx=0x7f2388008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:58.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.167+0000 7f2396a00640 1 -- 172.21.15.143:0/3356152361 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2390054af0 con 0x7f23a00b3890 2024-01-16T14:59:58.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.347+0000 7f23a4c00640 1 -- 172.21.15.143:0/3356152361 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23a00b33f0 con 0x7f23a00b3890 2024-01-16T14:59:58.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.350+0000 7f2396a00640 1 -- 172.21.15.143:0/3356152361 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12910 (secure 0 0 0) 0x7f2390017090 con 0x7f23a00b3890 2024-01-16T14:59:58.330 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:59:58.331 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":16,"fsid":"c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0","created":"2024-01-16T14:59:38.532601+0000","modified":"2024-01-16T14:59:54.038678+0000","last_up_change":"2024-01-16T14:59:50.983699+0000","last_in_change":"2024-01-16T14:59:42.056297+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:59:51.727720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f4c8891d-ab29-472a-907d-8446f20f7012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6801","nonce":1569293626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6805","nonce":1569293626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6814","nonce":1569293626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6809","nonce":1569293626}]},"public_addr":"172.21.15.143:6801/1569293626","cluster_addr":"172.21.15.143:6805/1569293626","heartbeat_back_addr":"172.21.15.143:6814/1569293626","heartbeat_front_addr":"172.21.15.143:6809/1569293626","state":["exists","up"]},{"osd":1,"uuid":"c4a2788d-8b89-4992-82b2-7f4b9debd509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6804","nonce":1276841168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6808","nonce":1276841168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6813","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6815","nonce":1276841168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6811","nonce":1276841168}]},"public_addr":"172.21.15.143:6804/1276841168","cluster_addr":"172.21.15.143:6808/1276841168","heartbeat_back_addr":"172.21.15.143:6815/1276841168","heartbeat_front_addr":"172.21.15.143:6811/1276841168","state":["exists","up"]},{"osd":2,"uuid":"64645c1d-31b2-4794-b0f2-7ccf8df30393","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6817","nonce":1217670170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6819","nonce":1217670170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6823","nonce":1217670170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6821","nonce":1217670170}]},"public_addr":"172.21.15.143:6817/1217670170","cluster_addr":"172.21.15.143:6819/1217670170","heartbeat_back_addr":"172.21.15.143:6823/1217670170","heartbeat_front_addr":"172.21.15.143:6821/1217670170","state":["exists","up"]},{"osd":3,"uuid":"73674129-a84e-4d31-8247-7d0179ae6df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6825","nonce":3414188012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6827","nonce":3414188012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6831","nonce":3414188012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6829","nonce":3414188012}]},"public_addr":"172.21.15.143:6825/3414188012","cluster_addr":"172.21.15.143:6827/3414188012","heartbeat_back_addr":"172.21.15.143:6831/3414188012","heartbeat_front_addr":"172.21.15.143:6829/3414188012","state":["exists","up"]},{"osd":4,"uuid":"cea28c9b-2f1c-4087-b415-9960789131d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6811","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6812","nonce":1070255716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6816","nonce":1070255716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6823","nonce":1070255716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6820","nonce":1070255716}]},"public_addr":"172.21.15.204:6812/1070255716","cluster_addr":"172.21.15.204:6816/1070255716","heartbeat_back_addr":"172.21.15.204:6823/1070255716","heartbeat_front_addr":"172.21.15.204:6820/1070255716","state":["exists","up"]},{"osd":5,"uuid":"33b63560-5574-4e78-983f-b398d67a2cb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6809","nonce":3605417837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6813","nonce":3605417837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6819","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6821","nonce":3605417837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6817","nonce":3605417837}]},"public_addr":"172.21.15.204:6809/3605417837","cluster_addr":"172.21.15.204:6813/3605417837","heartbeat_back_addr":"172.21.15.204:6821/3605417837","heartbeat_front_addr":"172.21.15.204:6817/3605417837","state":["exists","up"]},{"osd":6,"uuid":"df397427-8940-4eee-9c79-eece669d735d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6801","nonce":1085000534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6803","nonce":1085000534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6808","nonce":1085000534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6805","nonce":1085000534}]},"public_addr":"172.21.15.204:6801/1085000534","cluster_addr":"172.21.15.204:6803/1085000534","heartbeat_back_addr":"172.21.15.204:6808/1085000534","heartbeat_front_addr":"172.21.15.204:6805/1085000534","state":["exists","up"]},{"osd":7,"uuid":"a2e5a15c-8312-405a-ac02-f7e1fc9df2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6825","nonce":1337220609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6827","nonce":1337220609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6831","nonce":1337220609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6829","nonce":1337220609}]},"public_addr":"172.21.15.204:6825/1337220609","cluster_addr":"172.21.15.204:6827/1337220609","heartbeat_back_addr":"172.21.15.204:6831/1337220609","heartbeat_front_addr":"172.21.15.204:6829/1337220609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:49.984507+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.395306+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.632998+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.373627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.342920+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.438366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.331926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.406657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T14:59:58.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.353+0000 7f23a4c00640 1 -- 172.21.15.143:0/3356152361 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f237806b2f0 msgr2=0x7f237806d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:58.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.353+0000 7f23a4c00640 1 --2- 172.21.15.143:0/3356152361 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f237806b2f0 0x7f237806d7b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2388036000 tx=0x7f2388008030 comp rx=0 tx=0).stop 2024-01-16T14:59:58.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.353+0000 7f23a4c00640 1 -- 172.21.15.143:0/3356152361 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23a00b3890 msgr2=0x7f23a015c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:58.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.353+0000 7f23a4c00640 1 --2- 172.21.15.143:0/3356152361 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23a00b3890 0x7f23a015c220 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f239000e870 tx=0x7f239000ed30 comp rx=0 tx=0).stop 2024-01-16T14:59:58.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.353+0000 7f23a4c00640 1 -- 172.21.15.143:0/3356152361 shutdown_connections 2024-01-16T14:59:58.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.353+0000 7f23a4c00640 1 --2- 172.21.15.143:0/3356152361 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f237806b2f0 0x7f237806d7b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:58.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.353+0000 7f23a4c00640 1 --2- 172.21.15.143:0/3356152361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23a01461d0 0x7f23a00a9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:58.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.353+0000 7f23a4c00640 1 --2- 172.21.15.143:0/3356152361 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f23a00b4fe0 0x7f23a00cdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:58.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.353+0000 7f23a4c00640 1 --2- 172.21.15.143:0/3356152361 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23a00b3890 0x7f23a015c220 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:58.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.353+0000 7f23a4c00640 1 -- 172.21.15.143:0/3356152361 >> 172.21.15.143:0/3356152361 conn(0x7f23a0061340 msgr2=0x7f23a015dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:59:58.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.353+0000 7f23a4c00640 1 -- 172.21.15.143:0/3356152361 shutdown_connections 2024-01-16T14:59:58.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.353+0000 7f23a4c00640 1 -- 172.21.15.143:0/3356152361 wait complete. 2024-01-16T14:59:58.349 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-16T14:59:51.727720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-16T14:59:58.349 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-16T14:59:58.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.572+0000 7f863bc00640 1 Processor -- start 2024-01-16T14:59:58.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.573+0000 7f863bc00640 1 -- start start 2024-01-16T14:59:58.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.574+0000 7f863bc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86340d3a40 0x7f863406ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:58.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.574+0000 7f863bc00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f863415db00 con 0x7f86340b1ed0 2024-01-16T14:59:58.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.574+0000 7f863bc00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86340d0ce0 con 0x7f863415d720 2024-01-16T14:59:58.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.574+0000 7f863bc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f863406d400 con 0x7f86340d3a40 2024-01-16T14:59:58.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.574+0000 7f8639c00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f86340b1ed0 0x7f863406bfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55016/0 (socket says 172.21.15.143:55016) 2024-01-16T14:59:58.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.574+0000 7f8639c00640 1 -- 172.21.15.143:0/1964771889 learned_addr learned my addr 172.21.15.143:0/1964771889 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T14:59:58.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.574+0000 7f8633e00640 1 --2- 172.21.15.143:0/1964771889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86340d3a40 0x7f863406ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:58.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.575+0000 7f8633400640 1 -- 172.21.15.143:0/1964771889 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4055676156 0 0) 0x7f863415db00 con 0x7f86340b1ed0 2024-01-16T14:59:58.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.575+0000 7f8633400640 1 -- 172.21.15.143:0/1964771889 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8614003520 con 0x7f86340b1ed0 2024-01-16T14:59:58.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.575+0000 7f8633400640 1 -- 172.21.15.143:0/1964771889 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3216546958 0 0) 0x7f86340d0ce0 con 0x7f863415d720 2024-01-16T14:59:58.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.575+0000 7f8633400640 1 -- 172.21.15.143:0/1964771889 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f863415db00 con 0x7f863415d720 2024-01-16T14:59:58.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.575+0000 7f8633400640 1 -- 172.21.15.143:0/1964771889 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2687830753 0 0) 0x7f8614003520 con 0x7f86340b1ed0 2024-01-16T14:59:58.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.575+0000 7f8633400640 1 -- 172.21.15.143:0/1964771889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86340d3a40 msgr2=0x7f863406ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:59:58.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.575+0000 7f8633400640 1 --2- 172.21.15.143:0/1964771889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86340d3a40 0x7f863406ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:58.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.575+0000 7f8633400640 1 -- 172.21.15.143:0/1964771889 >> v1:172.21.15.204:6789/0 conn(0x7f863415d720 legacy=0x7f86340d3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:58.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.575+0000 7f8633400640 1 -- 172.21.15.143:0/1964771889 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86340b0100 con 0x7f86340b1ed0 2024-01-16T14:59:58.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.575+0000 7f8633400640 1 -- 172.21.15.143:0/1964771889 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7f862c001670 con 0x7f86340b1ed0 2024-01-16T14:59:58.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.575+0000 7f8633400640 1 -- 172.21.15.143:0/1964771889 >> v1:172.21.15.143:6789/0 conn(0x7f86340b1ed0 legacy=0x7f863406bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:58.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.575+0000 7f8633400640 1 --2- 172.21.15.143:0/1964771889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86340d3a40 0x7f8614003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:58.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.575+0000 7f8633400640 1 --2- 172.21.15.143:0/1964771889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8614004090 0x7f86140047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:58.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.575+0000 7f8633400640 1 --2- 172.21.15.143:0/1964771889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8614004d00 0x7f8614005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:58.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.575+0000 7f8633400640 1 -- 172.21.15.143:0/1964771889 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863406d400 con 0x7f8614004d00 2024-01-16T14:59:58.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.575+0000 7f8633400640 1 -- 172.21.15.143:0/1964771889 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f861400bb10 con 0x7f8614004090 2024-01-16T14:59:58.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.575+0000 7f8633400640 1 -- 172.21.15.143:0/1964771889 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f861400bca0 con 0x7f86340d3a40 2024-01-16T14:59:58.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.576+0000 7f8639c00640 1 --2- 172.21.15.143:0/1964771889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8614004d00 0x7f8614005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:58.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.576+0000 7f8633e00640 1 --2- 172.21.15.143:0/1964771889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8614004090 0x7f86140047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:58.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.576+0000 7f8639200640 1 --2- 172.21.15.143:0/1964771889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86340d3a40 0x7f8614003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:58.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.576+0000 7f8639c00640 1 -- 172.21.15.143:0/1964771889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86340d3a40 msgr2=0x7f8614003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:59:58.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.576+0000 7f8639c00640 1 --2- 172.21.15.143:0/1964771889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86340d3a40 0x7f8614003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:58.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.576+0000 7f8639c00640 1 -- 172.21.15.143:0/1964771889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8614004090 msgr2=0x7f86140047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:59:58.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.576+0000 7f8639c00640 1 --2- 172.21.15.143:0/1964771889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8614004090 0x7f86140047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:58.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.576+0000 7f8639c00640 1 -- 172.21.15.143:0/1964771889 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8614003520 con 0x7f8614004d00 2024-01-16T14:59:58.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.577+0000 7f8639c00640 1 --2- 172.21.15.143:0/1964771889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8614004d00 0x7f8614005180 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f862c007fd0 tx=0x7f86140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c6bfb4fd8fc3a2 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:58.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.577+0000 7f8633400640 1 -- 172.21.15.143:0/1964771889 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c005ad0 con 0x7f8614004d00 2024-01-16T14:59:58.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.577+0000 7f8633400640 1 -- 172.21.15.143:0/1964771889 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f862c005c50 con 0x7f8614004d00 2024-01-16T14:59:58.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.577+0000 7f8633400640 1 -- 172.21.15.143:0/1964771889 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c02c830 con 0x7f8614004d00 2024-01-16T14:59:58.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.578+0000 7f863bc00640 1 -- 172.21.15.143:0/1964771889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8614004d00 msgr2=0x7f8614005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:58.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.578+0000 7f863bc00640 1 --2- 172.21.15.143:0/1964771889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8614004d00 0x7f8614005180 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f862c007fd0 tx=0x7f86140057f0 comp rx=0 tx=0).stop 2024-01-16T14:59:58.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.578+0000 7f8639c00640 1 -- 172.21.15.143:0/1964771889 reap_dead start 2024-01-16T14:59:58.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.578+0000 7f863bc00640 1 -- 172.21.15.143:0/1964771889 shutdown_connections 2024-01-16T14:59:58.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.578+0000 7f863bc00640 1 -- 172.21.15.143:0/1964771889 >> 172.21.15.143:0/1964771889 conn(0x7f863408af60 msgr2=0x7f863409c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:59:58.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.578+0000 7f863bc00640 1 -- 172.21.15.143:0/1964771889 shutdown_connections 2024-01-16T14:59:58.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.578+0000 7f863bc00640 1 -- 172.21.15.143:0/1964771889 wait complete. 2024-01-16T14:59:58.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.580+0000 7f863bc00640 1 Processor -- start 2024-01-16T14:59:58.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.580+0000 7f863bc00640 1 -- start start 2024-01-16T14:59:58.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.581+0000 7f863bc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8614004d00 0x7f863415ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:58.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.581+0000 7f863bc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86340c6fe0 0x7f86340c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:58.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.581+0000 7f863bc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f863415bb10 0x7f863415bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:58.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.581+0000 7f863bc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863415c4d0 con 0x7f86340c6fe0 2024-01-16T14:59:58.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.581+0000 7f863bc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86340b8060 con 0x7f8614004d00 2024-01-16T14:59:58.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.581+0000 7f863bc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86340cafa0 con 0x7f863415bb10 2024-01-16T14:59:58.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.581+0000 7f8633e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86340c6fe0 0x7f86340c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:58.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.581+0000 7f8633e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86340c6fe0 0x7f86340c9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57018/0 (socket says 172.21.15.143:57018) 2024-01-16T14:59:58.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.581+0000 7f8633e00640 1 -- 172.21.15.143:0/4210728165 learned_addr learned my addr 172.21.15.143:0/4210728165 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T14:59:58.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.582+0000 7f8639200640 1 --2- 172.21.15.143:0/4210728165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8614004d00 0x7f863415ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:58.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.582+0000 7f8639c00640 1 --2- 172.21.15.143:0/4210728165 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f863415bb10 0x7f863415bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:58.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.582+0000 7f8633e00640 1 -- 172.21.15.143:0/4210728165 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f863415bb10 msgr2=0x7f863415bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:58.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.582+0000 7f8633e00640 1 --2- 172.21.15.143:0/4210728165 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f863415bb10 0x7f863415bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:58.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.582+0000 7f8633e00640 1 -- 172.21.15.143:0/4210728165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8614004d00 msgr2=0x7f863415ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:58.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.582+0000 7f8633e00640 1 --2- 172.21.15.143:0/4210728165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8614004d00 0x7f863415ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:58.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.582+0000 7f8633e00640 1 -- 172.21.15.143:0/4210728165 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86340cb280 con 0x7f86340c6fe0 2024-01-16T14:59:58.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.582+0000 7f8639c00640 1 --2- 172.21.15.143:0/4210728165 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f863415bb10 0x7f863415bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:59:58.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.582+0000 7f8633e00640 1 --2- 172.21.15.143:0/4210728165 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86340c6fe0 0x7f86340c9cb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8624009fd0 tx=0x7f86240115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:58.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.582+0000 7f8630c00640 1 -- 172.21.15.143:0/4210728165 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862400e030 con 0x7f86340c6fe0 2024-01-16T14:59:58.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.582+0000 7f8630c00640 1 -- 172.21.15.143:0/4210728165 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8624003990 con 0x7f86340c6fe0 2024-01-16T14:59:58.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.582+0000 7f8630c00640 1 -- 172.21.15.143:0/4210728165 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8624003b20 con 0x7f86340c6fe0 2024-01-16T14:59:58.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.582+0000 7f863bc00640 1 -- 172.21.15.143:0/4210728165 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86340cb560 con 0x7f86340c6fe0 2024-01-16T14:59:58.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.582+0000 7f863bc00640 1 -- 172.21.15.143:0/4210728165 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86340ca1f0 con 0x7f86340c6fe0 2024-01-16T14:59:58.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.583+0000 7f863bc00640 1 -- 172.21.15.143:0/4210728165 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86340d1f30 con 0x7f86340c6fe0 2024-01-16T14:59:58.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.585+0000 7f8630c00640 1 -- 172.21.15.143:0/4210728165 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f8624003c80 con 0x7f86340c6fe0 2024-01-16T14:59:58.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.585+0000 7f8630c00640 1 --2- 172.21.15.143:0/4210728165 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f860c06b310 0x7f860c06d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:58.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.586+0000 7f8630c00640 1 -- 172.21.15.143:0/4210728165 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f862400f070 con 0x7f86340c6fe0 2024-01-16T14:59:58.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.586+0000 7f8639200640 1 --2- 172.21.15.143:0/4210728165 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f860c06b310 0x7f860c06d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:58.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.586+0000 7f8639200640 1 --2- 172.21.15.143:0/4210728165 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f860c06b310 0x7f860c06d7d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f861c007fd0 tx=0x7f861c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:58.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.590+0000 7f8630c00640 1 -- 172.21.15.143:0/4210728165 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8624055610 con 0x7f86340c6fe0 2024-01-16T14:59:58.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.780+0000 7f863bc00640 1 -- 172.21.15.143:0/4210728165 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f863404e8b0 con 0x7f86340c6fe0 2024-01-16T14:59:58.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.781+0000 7f8630c00640 1 -- 172.21.15.143:0/4210728165 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8624058c30 con 0x7f86340c6fe0 2024-01-16T14:59:58.761 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-16T14:59:58.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.783+0000 7f863bc00640 1 -- 172.21.15.143:0/4210728165 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f860c06b310 msgr2=0x7f860c06d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:58.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.783+0000 7f863bc00640 1 --2- 172.21.15.143:0/4210728165 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f860c06b310 0x7f860c06d7d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f861c007fd0 tx=0x7f861c009040 comp rx=0 tx=0).stop 2024-01-16T14:59:58.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.783+0000 7f863bc00640 1 -- 172.21.15.143:0/4210728165 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86340c6fe0 msgr2=0x7f86340c9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:58.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.783+0000 7f863bc00640 1 --2- 172.21.15.143:0/4210728165 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86340c6fe0 0x7f86340c9cb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8624009fd0 tx=0x7f86240115a0 comp rx=0 tx=0).stop 2024-01-16T14:59:58.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.783+0000 7f863bc00640 1 -- 172.21.15.143:0/4210728165 shutdown_connections 2024-01-16T14:59:58.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.783+0000 7f863bc00640 1 --2- 172.21.15.143:0/4210728165 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f860c06b310 0x7f860c06d7d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:58.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.783+0000 7f863bc00640 1 --2- 172.21.15.143:0/4210728165 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f863415bb10 0x7f863415bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:58.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.783+0000 7f863bc00640 1 --2- 172.21.15.143:0/4210728165 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86340c6fe0 0x7f86340c9cb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:58.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.783+0000 7f863bc00640 1 --2- 172.21.15.143:0/4210728165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8614004d00 0x7f863415ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:58.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.783+0000 7f863bc00640 1 -- 172.21.15.143:0/4210728165 >> 172.21.15.143:0/4210728165 conn(0x7f863408af60 msgr2=0x7f86340b4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:59:58.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.784+0000 7f863bc00640 1 -- 172.21.15.143:0/4210728165 shutdown_connections 2024-01-16T14:59:58.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.784+0000 7f863bc00640 1 -- 172.21.15.143:0/4210728165 wait complete. 2024-01-16T14:59:58.783 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-16T14:59:58.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.992+0000 7f22ea400640 1 Processor -- start 2024-01-16T14:59:58.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.992+0000 7f22ea400640 1 -- start start 2024-01-16T14:59:58.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.993+0000 7f22ea400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f22e406d6c0 0x7f22e406db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:58.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.993+0000 7f22ea400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f22e40cdbf0 0x7f22e40ce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:58.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.993+0000 7f22ea400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e406e060 con 0x7f22e413d120 2024-01-16T14:59:58.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.994+0000 7f22ea400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e413f9e0 con 0x7f22e406d6c0 2024-01-16T14:59:58.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.994+0000 7f22ea400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e413fcc0 con 0x7f22e40cdbf0 2024-01-16T14:59:58.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.994+0000 7f22e9000640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f22e413d120 0x7f22e415af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:44932/0 (socket says 172.21.15.143:44932) 2024-01-16T14:59:58.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.994+0000 7f22e9000640 1 -- 172.21.15.143:0/3022703566 learned_addr learned my addr 172.21.15.143:0/3022703566 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T14:59:58.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.994+0000 7f22e9a00640 1 --2- 172.21.15.143:0/3022703566 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f22e40cdbf0 0x7f22e40ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:58.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.994+0000 7f22e3e00640 1 --2- 172.21.15.143:0/3022703566 >> v2:172.21.15.204:3300/0 conn(0x7f22e406d6c0 0x7f22e406db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:58.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.995+0000 7f22e3400640 1 -- 172.21.15.143:0/3022703566 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1835069072 0 0) 0x7f22e406e060 con 0x7f22e413d120 2024-01-16T14:59:58.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.995+0000 7f22e3400640 1 -- 172.21.15.143:0/3022703566 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22c4003520 con 0x7f22e413d120 2024-01-16T14:59:58.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.995+0000 7f22e3e00640 1 -- 172.21.15.143:0/3022703566 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f22e40cdbf0 msgr2=0x7f22e40ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:59:58.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.995+0000 7f22e3e00640 1 --2- 172.21.15.143:0/3022703566 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f22e40cdbf0 0x7f22e40ce060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:58.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.995+0000 7f22e3e00640 1 -- 172.21.15.143:0/3022703566 >> v1:172.21.15.204:6789/0 conn(0x7f22e413d120 legacy=0x7f22e415af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:58.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.995+0000 7f22e3e00640 1 -- 172.21.15.143:0/3022703566 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e413ffa0 con 0x7f22e406d6c0 2024-01-16T14:59:58.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.997+0000 7f22e3e00640 1 --2- 172.21.15.143:0/3022703566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22e406d6c0 0x7f22e406db20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f22cc0097a0 tx=0x7f22cc02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32255ac61c4045f8 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:58.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.997+0000 7f22e3400640 1 -- 172.21.15.143:0/3022703566 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc02e070 con 0x7f22e406d6c0 2024-01-16T14:59:58.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.997+0000 7f22e3400640 1 -- 172.21.15.143:0/3022703566 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22cc0315a0 con 0x7f22e406d6c0 2024-01-16T14:59:58.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.997+0000 7f22e3400640 1 -- 172.21.15.143:0/3022703566 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc031930 con 0x7f22e406d6c0 2024-01-16T14:59:58.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.998+0000 7f22ea400640 1 -- 172.21.15.143:0/3022703566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22e406d6c0 msgr2=0x7f22e406db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:58.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.998+0000 7f22ea400640 1 --2- 172.21.15.143:0/3022703566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22e406d6c0 0x7f22e406db20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f22cc0097a0 tx=0x7f22cc02b840 comp rx=0 tx=0).stop 2024-01-16T14:59:58.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.998+0000 7f22ea400640 1 -- 172.21.15.143:0/3022703566 shutdown_connections 2024-01-16T14:59:58.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.998+0000 7f22ea400640 1 --2- 172.21.15.143:0/3022703566 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f22e40cdbf0 0x7f22e40ce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:58.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.998+0000 7f22ea400640 1 --2- 172.21.15.143:0/3022703566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22e406d6c0 0x7f22e406db20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:58.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.998+0000 7f22ea400640 1 -- 172.21.15.143:0/3022703566 >> 172.21.15.143:0/3022703566 conn(0x7f22e40cb830 msgr2=0x7f22e40cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:59:58.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.998+0000 7f22ea400640 1 -- 172.21.15.143:0/3022703566 shutdown_connections 2024-01-16T14:59:58.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:58.998+0000 7f22ea400640 1 -- 172.21.15.143:0/3022703566 wait complete. 2024-01-16T14:59:58.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.000+0000 7f22ea400640 1 Processor -- start 2024-01-16T14:59:58.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.000+0000 7f22ea400640 1 -- start start 2024-01-16T14:59:58.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.001+0000 7f22ea400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22e406d6c0 0x7f22e413b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:58.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.001+0000 7f22ea400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22e40cdbf0 0x7f22e4152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:58.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.001+0000 7f22ea400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f22e413d120 0x7f22e40b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:58.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.001+0000 7f22ea400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e4151680 con 0x7f22e40cdbf0 2024-01-16T14:59:58.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.001+0000 7f22ea400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e4151960 con 0x7f22e406d6c0 2024-01-16T14:59:58.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.001+0000 7f22ea400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e4151c40 con 0x7f22e413d120 2024-01-16T14:59:58.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.001+0000 7f22e3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22e40cdbf0 0x7f22e4152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:58.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.002+0000 7f22e3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22e40cdbf0 0x7f22e4152cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57024/0 (socket says 172.21.15.143:57024) 2024-01-16T14:59:58.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.002+0000 7f22e3e00640 1 -- 172.21.15.143:0/2907422179 learned_addr learned my addr 172.21.15.143:0/2907422179 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T14:59:58.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.002+0000 7f22e9000640 1 --2- 172.21.15.143:0/2907422179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22e406d6c0 0x7f22e413b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:58.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.002+0000 7f22e9a00640 1 --2- 172.21.15.143:0/2907422179 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f22e413d120 0x7f22e40b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:58.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.002+0000 7f22e3e00640 1 -- 172.21.15.143:0/2907422179 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f22e413d120 msgr2=0x7f22e40b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:58.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.002+0000 7f22e3e00640 1 --2- 172.21.15.143:0/2907422179 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f22e413d120 0x7f22e40b3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:58.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.002+0000 7f22e3e00640 1 -- 172.21.15.143:0/2907422179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22e406d6c0 msgr2=0x7f22e413b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:58.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.002+0000 7f22e3e00640 1 --2- 172.21.15.143:0/2907422179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22e406d6c0 0x7f22e413b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:58.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.002+0000 7f22e3e00640 1 -- 172.21.15.143:0/2907422179 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22cc009400 con 0x7f22e40cdbf0 2024-01-16T14:59:58.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.002+0000 7f22e3e00640 1 --2- 172.21.15.143:0/2907422179 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22e40cdbf0 0x7f22e4152cf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f22cc034040 tx=0x7f22cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:58.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.003+0000 7f22e0c00640 1 -- 172.21.15.143:0/2907422179 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc032630 con 0x7f22e40cdbf0 2024-01-16T14:59:58.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.003+0000 7f22e0c00640 1 -- 172.21.15.143:0/2907422179 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22cc032790 con 0x7f22e40cdbf0 2024-01-16T14:59:58.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.003+0000 7f22e0c00640 1 -- 172.21.15.143:0/2907422179 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc032b40 con 0x7f22e40cdbf0 2024-01-16T14:59:58.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.003+0000 7f22ea400640 1 -- 172.21.15.143:0/2907422179 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e415a9d0 con 0x7f22e40cdbf0 2024-01-16T14:59:58.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.003+0000 7f22ea400640 1 -- 172.21.15.143:0/2907422179 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e4152200 con 0x7f22e40cdbf0 2024-01-16T14:59:58.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.004+0000 7f22ea400640 1 -- 172.21.15.143:0/2907422179 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e404e840 con 0x7f22e40cdbf0 2024-01-16T14:59:58.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.005+0000 7f22e0c00640 1 -- 172.21.15.143:0/2907422179 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f22cc04e430 con 0x7f22e40cdbf0 2024-01-16T14:59:58.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.006+0000 7f22e0c00640 1 --2- 172.21.15.143:0/2907422179 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f22bc06b310 0x7f22bc06d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:58.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.006+0000 7f22e0c00640 1 -- 172.21.15.143:0/2907422179 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f22cc0ab8e0 con 0x7f22e40cdbf0 2024-01-16T14:59:58.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.006+0000 7f22e9000640 1 --2- 172.21.15.143:0/2907422179 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f22bc06b310 0x7f22bc06d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:58.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.006+0000 7f22e9000640 1 --2- 172.21.15.143:0/2907422179 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f22bc06b310 0x7f22bc06d7d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f22d40011e0 tx=0x7f22d4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:58.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.010+0000 7f22e0c00640 1 -- 172.21.15.143:0/2907422179 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f22cc076af0 con 0x7f22e40cdbf0 2024-01-16T14:59:59.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.221+0000 7f22ea400640 1 -- 172.21.15.143:0/2907422179 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22e4159d50 con 0x7f22e40cdbf0 2024-01-16T14:59:59.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.222+0000 7f22e0c00640 1 -- 172.21.15.143:0/2907422179 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3703 (secure 0 0 0) 0x7f22cc07a110 con 0x7f22e40cdbf0 2024-01-16T14:59:59.203 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:59:59.203 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3599169408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3599169408},{"type":"v1","addr":"172.21.15.204:6834","nonce":3599169408}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/859082411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":859082411},{"type":"v1","addr":"172.21.15.204:6835","nonce":859082411}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6836/1564738611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1564738611},{"type":"v1","addr":"172.21.15.143:6836","nonce":1564738611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/3120706283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6835","nonce":3120706283},{"type":"v1","addr":"172.21.15.143:6837","nonce":3120706283}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-01-16T14:59:59.203 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 3 2024-01-16T14:59:59.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.225+0000 7f22ea400640 1 -- 172.21.15.143:0/2907422179 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f22bc06b310 msgr2=0x7f22bc06d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:59.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.225+0000 7f22ea400640 1 --2- 172.21.15.143:0/2907422179 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f22bc06b310 0x7f22bc06d7d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f22d40011e0 tx=0x7f22d4009040 comp rx=0 tx=0).stop 2024-01-16T14:59:59.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.225+0000 7f22ea400640 1 -- 172.21.15.143:0/2907422179 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22e40cdbf0 msgr2=0x7f22e4152cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:59.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.225+0000 7f22ea400640 1 --2- 172.21.15.143:0/2907422179 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22e40cdbf0 0x7f22e4152cf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f22cc034040 tx=0x7f22cc0046c0 comp rx=0 tx=0).stop 2024-01-16T14:59:59.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.225+0000 7f22ea400640 1 -- 172.21.15.143:0/2907422179 shutdown_connections 2024-01-16T14:59:59.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.225+0000 7f22ea400640 1 --2- 172.21.15.143:0/2907422179 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f22bc06b310 0x7f22bc06d7d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:59.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.225+0000 7f22ea400640 1 --2- 172.21.15.143:0/2907422179 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f22e413d120 0x7f22e40b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:59.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.225+0000 7f22ea400640 1 --2- 172.21.15.143:0/2907422179 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22e40cdbf0 0x7f22e4152cf0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:59.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.225+0000 7f22ea400640 1 --2- 172.21.15.143:0/2907422179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22e406d6c0 0x7f22e413b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:59.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.225+0000 7f22ea400640 1 -- 172.21.15.143:0/2907422179 >> 172.21.15.143:0/2907422179 conn(0x7f22e40cb830 msgr2=0x7f22e40cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:59:59.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.225+0000 7f22ea400640 1 -- 172.21.15.143:0/2907422179 shutdown_connections 2024-01-16T14:59:59.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.225+0000 7f22ea400640 1 -- 172.21.15.143:0/2907422179 wait complete. 2024-01-16T14:59:59.223 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T14:59:59.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.444+0000 7fbac1600640 1 Processor -- start 2024-01-16T14:59:59.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.444+0000 7fbac1600640 1 -- start start 2024-01-16T14:59:59.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.445+0000 7fbac1600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbabc159ce0 0x7fbabc06de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:59.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.446+0000 7fbac1600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbabc0cc150 0x7fbabc06c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:59.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.446+0000 7fbac1600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbabc0b3a40 con 0x7fbabc0b21d0 2024-01-16T14:59:59.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.446+0000 7fbac1600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbabc0cdfa0 con 0x7fbabc0cc150 2024-01-16T14:59:59.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.446+0000 7fbac1600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbabc0cd9e0 con 0x7fbabc159ce0 2024-01-16T14:59:59.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.446+0000 7fbabb400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbabc0cc150 0x7fbabc06c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:59.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.446+0000 7fbabb400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbabc0cc150 0x7fbabc06c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57034/0 (socket says 172.21.15.143:57034) 2024-01-16T14:59:59.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.446+0000 7fbabb400640 1 -- 172.21.15.143:0/753836562 learned_addr learned my addr 172.21.15.143:0/753836562 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T14:59:59.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.446+0000 7fbabaa00640 1 --2- 172.21.15.143:0/753836562 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbabc159ce0 0x7fbabc06de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:59.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.446+0000 7fbabb400640 1 -- 172.21.15.143:0/753836562 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbabc159ce0 msgr2=0x7fbabc06de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:59:59.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.446+0000 7fbabb400640 1 --2- 172.21.15.143:0/753836562 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbabc159ce0 0x7fbabc06de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:59.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.446+0000 7fbabb400640 1 -- 172.21.15.143:0/753836562 >> v1:172.21.15.204:6789/0 conn(0x7fbabc0b21d0 legacy=0x7fbabc0cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:59:59.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.446+0000 7fbabb400640 1 -- 172.21.15.143:0/753836562 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbabc0cdcc0 con 0x7fbabc0cc150 2024-01-16T14:59:59.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.447+0000 7fbabb400640 1 --2- 172.21.15.143:0/753836562 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbabc0cc150 0x7fbabc06c560 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbab400d150 tx=0x7fbab402e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d603431088464048 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:59.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.448+0000 7fbaba000640 1 -- 172.21.15.143:0/753836562 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab402ea80 con 0x7fbabc0cc150 2024-01-16T14:59:59.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.448+0000 7fbaba000640 1 -- 172.21.15.143:0/753836562 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbab402ebe0 con 0x7fbabc0cc150 2024-01-16T14:59:59.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.448+0000 7fbaba000640 1 -- 172.21.15.143:0/753836562 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab4009600 con 0x7fbabc0cc150 2024-01-16T14:59:59.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.448+0000 7fbac1600640 1 -- 172.21.15.143:0/753836562 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbabc0cc150 msgr2=0x7fbabc06c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:59.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.448+0000 7fbac1600640 1 --2- 172.21.15.143:0/753836562 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbabc0cc150 0x7fbabc06c560 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbab400d150 tx=0x7fbab402e0e0 comp rx=0 tx=0).stop 2024-01-16T14:59:59.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.448+0000 7fbac1600640 1 -- 172.21.15.143:0/753836562 shutdown_connections 2024-01-16T14:59:59.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.448+0000 7fbac1600640 1 --2- 172.21.15.143:0/753836562 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbabc0cc150 0x7fbabc06c560 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:59.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.448+0000 7fbac1600640 1 --2- 172.21.15.143:0/753836562 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbabc159ce0 0x7fbabc06de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:59.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.448+0000 7fbac1600640 1 -- 172.21.15.143:0/753836562 >> 172.21.15.143:0/753836562 conn(0x7fbabc06a920 msgr2=0x7fbabc06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:59:59.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.448+0000 7fbac1600640 1 -- 172.21.15.143:0/753836562 shutdown_connections 2024-01-16T14:59:59.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.448+0000 7fbac1600640 1 -- 172.21.15.143:0/753836562 wait complete. 2024-01-16T14:59:59.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.450+0000 7fbac1600640 1 Processor -- start 2024-01-16T14:59:59.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.450+0000 7fbac1600640 1 -- start start 2024-01-16T14:59:59.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.451+0000 7fbac1600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbabc0b21d0 0x7fbabc099140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:59.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.451+0000 7fbac1600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbabc0cc150 0x7fbabc09abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:59.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.451+0000 7fbac1600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbabc159ce0 0x7fbabc1f2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:59.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.451+0000 7fbac1600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbabc09b100 con 0x7fbabc0b21d0 2024-01-16T14:59:59.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.451+0000 7fbac1600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbabc1456c0 con 0x7fbabc0cc150 2024-01-16T14:59:59.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.451+0000 7fbac1600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbabc144ab0 con 0x7fbabc159ce0 2024-01-16T14:59:59.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.451+0000 7fbabaa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbabc0b21d0 0x7fbabc099140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:59.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.451+0000 7fbabaa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbabc0b21d0 0x7fbabc099140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57042/0 (socket says 172.21.15.143:57042) 2024-01-16T14:59:59.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.451+0000 7fbabaa00640 1 -- 172.21.15.143:0/1300341865 learned_addr learned my addr 172.21.15.143:0/1300341865 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T14:59:59.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.452+0000 7fbabaa00640 1 -- 172.21.15.143:0/1300341865 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbabc159ce0 msgr2=0x7fbabc1f2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:59.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.452+0000 7fbabaa00640 1 --2- 172.21.15.143:0/1300341865 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbabc159ce0 0x7fbabc1f2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:59.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.452+0000 7fbab1e00640 1 --2- 172.21.15.143:0/1300341865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbabc0cc150 0x7fbabc09abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:59.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.452+0000 7fbabaa00640 1 -- 172.21.15.143:0/1300341865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbabc0cc150 msgr2=0x7fbabc09abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:59.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.452+0000 7fbabaa00640 1 --2- 172.21.15.143:0/1300341865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbabc0cc150 0x7fbabc09abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:59.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.452+0000 7fbabaa00640 1 -- 172.21.15.143:0/1300341865 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab400bd40 con 0x7fbabc0b21d0 2024-01-16T14:59:59.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.452+0000 7fbab1e00640 1 --2- 172.21.15.143:0/1300341865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbabc0cc150 0x7fbabc09abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:59:59.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.452+0000 7fbabaa00640 1 --2- 172.21.15.143:0/1300341865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbabc0b21d0 0x7fbabc099140 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbaac00c7f0 tx=0x7fbaac00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:59.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.452+0000 7fbab3400640 1 -- 172.21.15.143:0/1300341865 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaac004220 con 0x7fbabc0b21d0 2024-01-16T14:59:59.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.452+0000 7fbab3400640 1 -- 172.21.15.143:0/1300341865 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaac004380 con 0x7fbabc0b21d0 2024-01-16T14:59:59.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.452+0000 7fbab3400640 1 -- 172.21.15.143:0/1300341865 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaac007840 con 0x7fbabc0b21d0 2024-01-16T14:59:59.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.452+0000 7fbac1600640 1 -- 172.21.15.143:0/1300341865 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbabc144d90 con 0x7fbabc0b21d0 2024-01-16T14:59:59.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.453+0000 7fbac1600640 1 -- 172.21.15.143:0/1300341865 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbabc145350 con 0x7fbabc0b21d0 2024-01-16T14:59:59.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.454+0000 7fbac1600640 1 -- 172.21.15.143:0/1300341865 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbabc06e370 con 0x7fbabc0b21d0 2024-01-16T14:59:59.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.454+0000 7fbab3400640 1 -- 172.21.15.143:0/1300341865 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fbaac007bb0 con 0x7fbabc0b21d0 2024-01-16T14:59:59.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.455+0000 7fbab3400640 1 --2- 172.21.15.143:0/1300341865 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fba9406b2f0 0x7fba9406d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:59.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.455+0000 7fbab3400640 1 -- 172.21.15.143:0/1300341865 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbaac089700 con 0x7fbabc0b21d0 2024-01-16T14:59:59.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.456+0000 7fbab1e00640 1 --2- 172.21.15.143:0/1300341865 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fba9406b2f0 0x7fba9406d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:59.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.456+0000 7fbab1e00640 1 --2- 172.21.15.143:0/1300341865 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fba9406b2f0 0x7fba9406d7b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbabc1f6830 tx=0x7fbaa4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:59.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.460+0000 7fbab3400640 1 -- 172.21.15.143:0/1300341865 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbaac0548c0 con 0x7fbabc0b21d0 2024-01-16T14:59:59.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.656+0000 7fbac1600640 1 -- 172.21.15.143:0/1300341865 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbabc067470 con 0x7fbabc0b21d0 2024-01-16T14:59:59.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.657+0000 7fbab3400640 1 -- 172.21.15.143:0/1300341865 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12910 (secure 0 0 0) 0x7fbaac01e090 con 0x7fbabc0b21d0 2024-01-16T14:59:59.638 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T14:59:59.638 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":16,"fsid":"c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0","created":"2024-01-16T14:59:38.532601+0000","modified":"2024-01-16T14:59:54.038678+0000","last_up_change":"2024-01-16T14:59:50.983699+0000","last_in_change":"2024-01-16T14:59:42.056297+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:59:51.727720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f4c8891d-ab29-472a-907d-8446f20f7012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6801","nonce":1569293626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6805","nonce":1569293626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6814","nonce":1569293626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6809","nonce":1569293626}]},"public_addr":"172.21.15.143:6801/1569293626","cluster_addr":"172.21.15.143:6805/1569293626","heartbeat_back_addr":"172.21.15.143:6814/1569293626","heartbeat_front_addr":"172.21.15.143:6809/1569293626","state":["exists","up"]},{"osd":1,"uuid":"c4a2788d-8b89-4992-82b2-7f4b9debd509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6804","nonce":1276841168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6808","nonce":1276841168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6813","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6815","nonce":1276841168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6811","nonce":1276841168}]},"public_addr":"172.21.15.143:6804/1276841168","cluster_addr":"172.21.15.143:6808/1276841168","heartbeat_back_addr":"172.21.15.143:6815/1276841168","heartbeat_front_addr":"172.21.15.143:6811/1276841168","state":["exists","up"]},{"osd":2,"uuid":"64645c1d-31b2-4794-b0f2-7ccf8df30393","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6817","nonce":1217670170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6819","nonce":1217670170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6823","nonce":1217670170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6821","nonce":1217670170}]},"public_addr":"172.21.15.143:6817/1217670170","cluster_addr":"172.21.15.143:6819/1217670170","heartbeat_back_addr":"172.21.15.143:6823/1217670170","heartbeat_front_addr":"172.21.15.143:6821/1217670170","state":["exists","up"]},{"osd":3,"uuid":"73674129-a84e-4d31-8247-7d0179ae6df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6825","nonce":3414188012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6827","nonce":3414188012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6831","nonce":3414188012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6829","nonce":3414188012}]},"public_addr":"172.21.15.143:6825/3414188012","cluster_addr":"172.21.15.143:6827/3414188012","heartbeat_back_addr":"172.21.15.143:6831/3414188012","heartbeat_front_addr":"172.21.15.143:6829/3414188012","state":["exists","up"]},{"osd":4,"uuid":"cea28c9b-2f1c-4087-b415-9960789131d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6811","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6812","nonce":1070255716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6816","nonce":1070255716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6823","nonce":1070255716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6820","nonce":1070255716}]},"public_addr":"172.21.15.204:6812/1070255716","cluster_addr":"172.21.15.204:6816/1070255716","heartbeat_back_addr":"172.21.15.204:6823/1070255716","heartbeat_front_addr":"172.21.15.204:6820/1070255716","state":["exists","up"]},{"osd":5,"uuid":"33b63560-5574-4e78-983f-b398d67a2cb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6809","nonce":3605417837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6813","nonce":3605417837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6819","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6821","nonce":3605417837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6817","nonce":3605417837}]},"public_addr":"172.21.15.204:6809/3605417837","cluster_addr":"172.21.15.204:6813/3605417837","heartbeat_back_addr":"172.21.15.204:6821/3605417837","heartbeat_front_addr":"172.21.15.204:6817/3605417837","state":["exists","up"]},{"osd":6,"uuid":"df397427-8940-4eee-9c79-eece669d735d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6801","nonce":1085000534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6803","nonce":1085000534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6808","nonce":1085000534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6805","nonce":1085000534}]},"public_addr":"172.21.15.204:6801/1085000534","cluster_addr":"172.21.15.204:6803/1085000534","heartbeat_back_addr":"172.21.15.204:6808/1085000534","heartbeat_front_addr":"172.21.15.204:6805/1085000534","state":["exists","up"]},{"osd":7,"uuid":"a2e5a15c-8312-405a-ac02-f7e1fc9df2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6825","nonce":1337220609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6827","nonce":1337220609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6831","nonce":1337220609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6829","nonce":1337220609}]},"public_addr":"172.21.15.204:6825/1337220609","cluster_addr":"172.21.15.204:6827/1337220609","heartbeat_back_addr":"172.21.15.204:6831/1337220609","heartbeat_front_addr":"172.21.15.204:6829/1337220609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:49.984507+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.395306+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.632998+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.373627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.342920+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.438366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.331926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.406657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T14:59:59.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.660+0000 7fbac1600640 1 -- 172.21.15.143:0/1300341865 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fba9406b2f0 msgr2=0x7fba9406d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:59.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.660+0000 7fbac1600640 1 --2- 172.21.15.143:0/1300341865 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fba9406b2f0 0x7fba9406d7b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbabc1f6830 tx=0x7fbaa4009040 comp rx=0 tx=0).stop 2024-01-16T14:59:59.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.660+0000 7fbac1600640 1 -- 172.21.15.143:0/1300341865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbabc0b21d0 msgr2=0x7fbabc099140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:59.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.660+0000 7fbac1600640 1 --2- 172.21.15.143:0/1300341865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbabc0b21d0 0x7fbabc099140 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbaac00c7f0 tx=0x7fbaac00ccb0 comp rx=0 tx=0).stop 2024-01-16T14:59:59.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.660+0000 7fbac1600640 1 -- 172.21.15.143:0/1300341865 shutdown_connections 2024-01-16T14:59:59.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.660+0000 7fbac1600640 1 --2- 172.21.15.143:0/1300341865 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fba9406b2f0 0x7fba9406d7b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:59.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.660+0000 7fbac1600640 1 --2- 172.21.15.143:0/1300341865 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbabc159ce0 0x7fbabc1f2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:59.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.660+0000 7fbac1600640 1 --2- 172.21.15.143:0/1300341865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbabc0cc150 0x7fbabc09abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:59.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.660+0000 7fbac1600640 1 --2- 172.21.15.143:0/1300341865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbabc0b21d0 0x7fbabc099140 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:59.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.660+0000 7fbac1600640 1 -- 172.21.15.143:0/1300341865 >> 172.21.15.143:0/1300341865 conn(0x7fbabc06a920 msgr2=0x7fbabc0ad910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:59:59.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.661+0000 7fbac1600640 1 -- 172.21.15.143:0/1300341865 shutdown_connections 2024-01-16T14:59:59.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.661+0000 7fbac1600640 1 -- 172.21.15.143:0/1300341865 wait complete. 2024-01-16T14:59:59.659 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-16T14:59:51.727720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-16T14:59:59.659 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-16T14:59:59.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.886+0000 7f6c15600640 1 Processor -- start 2024-01-16T14:59:59.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.886+0000 7f6c15600640 1 -- start start 2024-01-16T14:59:59.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.888+0000 7f6c15600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6c1013e3e0 0x7f6c100ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:59.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.888+0000 7f6c15600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6c1013ea00 0x7f6c100cede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:59.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.888+0000 7f6c15600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c1015c630 con 0x7f6c100cf320 2024-01-16T14:59:59.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.888+0000 7f6c15600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c1013e7c0 con 0x7f6c1013ea00 2024-01-16T14:59:59.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.888+0000 7f6c15600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c1015b810 con 0x7f6c1013e3e0 2024-01-16T14:59:59.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.888+0000 7f6c0e000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6c1013ea00 0x7f6c100cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:59.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.888+0000 7f6c0f400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f6c100cf320 0x7f6c100700b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55020/0 (socket says 172.21.15.143:55020) 2024-01-16T14:59:59.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.888+0000 7f6c0f400640 1 -- 172.21.15.143:0/4268468715 learned_addr learned my addr 172.21.15.143:0/4268468715 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T14:59:59.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.888+0000 7f6c0ea00640 1 --2- 172.21.15.143:0/4268468715 >> v2:172.21.15.204:3300/0 conn(0x7f6c1013e3e0 0x7f6c100ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:59.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.889+0000 7f6c0e000640 1 -- 172.21.15.143:0/4268468715 >> v2:172.21.15.204:3300/0 conn(0x7f6c1013e3e0 msgr2=0x7f6c100ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:59:59.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.889+0000 7f6c0e000640 1 --2- 172.21.15.143:0/4268468715 >> v2:172.21.15.204:3300/0 conn(0x7f6c1013e3e0 0x7f6c100ce3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:59.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.889+0000 7f6c0e000640 1 -- 172.21.15.143:0/4268468715 >> v1:172.21.15.143:6789/0 conn(0x7f6c100cf320 legacy=0x7f6c100700b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:59.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.889+0000 7f6c0e000640 1 -- 172.21.15.143:0/4268468715 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1015ba20 con 0x7f6c1013ea00 2024-01-16T14:59:59.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.889+0000 7f6c0d600640 1 -- 172.21.15.143:0/4268468715 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3331864378 0 0) 0x7f6c1015c630 con 0x7f6c100cf320 2024-01-16T14:59:59.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.890+0000 7f6c0e000640 1 --2- 172.21.15.143:0/4268468715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c1013ea00 0x7f6c100cede0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6bf80097a0 tx=0x7f6bf802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be82ac77f00a134b server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:59.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.890+0000 7f6c0d600640 1 -- 172.21.15.143:0/4268468715 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf802e070 con 0x7f6c1013ea00 2024-01-16T14:59:59.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.890+0000 7f6c0d600640 1 -- 172.21.15.143:0/4268468715 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf80315a0 con 0x7f6c1013ea00 2024-01-16T14:59:59.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.890+0000 7f6c0d600640 1 -- 172.21.15.143:0/4268468715 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf8031930 con 0x7f6c1013ea00 2024-01-16T14:59:59.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.891+0000 7f6c15600640 1 -- 172.21.15.143:0/4268468715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c1013ea00 msgr2=0x7f6c100cede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:59.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.891+0000 7f6c15600640 1 --2- 172.21.15.143:0/4268468715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c1013ea00 0x7f6c100cede0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6bf80097a0 tx=0x7f6bf802b840 comp rx=0 tx=0).stop 2024-01-16T14:59:59.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.891+0000 7f6c15600640 1 -- 172.21.15.143:0/4268468715 shutdown_connections 2024-01-16T14:59:59.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.891+0000 7f6c15600640 1 --2- 172.21.15.143:0/4268468715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c1013ea00 0x7f6c100cede0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:59.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.891+0000 7f6c15600640 1 --2- 172.21.15.143:0/4268468715 >> v2:172.21.15.204:3300/0 conn(0x7f6c1013e3e0 0x7f6c100ce3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:59.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.891+0000 7f6c15600640 1 -- 172.21.15.143:0/4268468715 >> 172.21.15.143:0/4268468715 conn(0x7f6c100b2e40 msgr2=0x7f6c100b3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:59:59.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.891+0000 7f6c15600640 1 -- 172.21.15.143:0/4268468715 shutdown_connections 2024-01-16T14:59:59.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.891+0000 7f6c15600640 1 -- 172.21.15.143:0/4268468715 wait complete. 2024-01-16T14:59:59.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.893+0000 7f6c15600640 1 Processor -- start 2024-01-16T14:59:59.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.893+0000 7f6c15600640 1 -- start start 2024-01-16T14:59:59.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.894+0000 7f6c15600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c100cf320 0x7f6c100c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:59.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.894+0000 7f6c15600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c1013e3e0 0x7f6c100b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:59.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.894+0000 7f6c15600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c1013ea00 0x7f6c10154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:59.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.894+0000 7f6c15600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c10159430 con 0x7f6c1013ea00 2024-01-16T14:59:59.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.894+0000 7f6c15600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c10159610 con 0x7f6c100cf320 2024-01-16T14:59:59.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.894+0000 7f6c15600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c101598f0 con 0x7f6c1013e3e0 2024-01-16T14:59:59.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.894+0000 7f6c0f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c1013ea00 0x7f6c10154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:59.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.894+0000 7f6c0f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c1013ea00 0x7f6c10154480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57060/0 (socket says 172.21.15.143:57060) 2024-01-16T14:59:59.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.894+0000 7f6c0f400640 1 -- 172.21.15.143:0/3505019480 learned_addr learned my addr 172.21.15.143:0/3505019480 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T14:59:59.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.894+0000 7f6c0ea00640 1 --2- 172.21.15.143:0/3505019480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c100cf320 0x7f6c100c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:59.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.894+0000 7f6c0e000640 1 --2- 172.21.15.143:0/3505019480 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c1013e3e0 0x7f6c100b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:59.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.895+0000 7f6c0f400640 1 -- 172.21.15.143:0/3505019480 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c1013e3e0 msgr2=0x7f6c100b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:59.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.895+0000 7f6c0f400640 1 --2- 172.21.15.143:0/3505019480 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c1013e3e0 0x7f6c100b5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:59.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.895+0000 7f6c0f400640 1 -- 172.21.15.143:0/3505019480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c100cf320 msgr2=0x7f6c100c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:59:59.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.895+0000 7f6c0f400640 1 --2- 172.21.15.143:0/3505019480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c100cf320 0x7f6c100c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:59:59.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.895+0000 7f6c0f400640 1 -- 172.21.15.143:0/3505019480 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf8009400 con 0x7f6c1013ea00 2024-01-16T14:59:59.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.895+0000 7f6c0f400640 1 --2- 172.21.15.143:0/3505019480 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c1013ea00 0x7f6c10154480 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6c0800c780 tx=0x7f6c0800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:59.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.895+0000 7f6c06a00640 1 -- 172.21.15.143:0/3505019480 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0800ce90 con 0x7f6c1013ea00 2024-01-16T14:59:59.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.895+0000 7f6c06a00640 1 -- 172.21.15.143:0/3505019480 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c0800a640 con 0x7f6c1013ea00 2024-01-16T14:59:59.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.895+0000 7f6c06a00640 1 -- 172.21.15.143:0/3505019480 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0800a9d0 con 0x7f6c1013ea00 2024-01-16T14:59:59.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.896+0000 7f6c15600640 1 -- 172.21.15.143:0/3505019480 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c10159bd0 con 0x7f6c1013ea00 2024-01-16T14:59:59.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.896+0000 7f6c15600640 1 -- 172.21.15.143:0/3505019480 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c1015a190 con 0x7f6c1013ea00 2024-01-16T14:59:59.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.897+0000 7f6c15600640 1 -- 172.21.15.143:0/3505019480 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c1015a380 con 0x7f6c1013ea00 2024-01-16T14:59:59.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.898+0000 7f6c06a00640 1 -- 172.21.15.143:0/3505019480 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f6c0800ab30 con 0x7f6c1013ea00 2024-01-16T14:59:59.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.899+0000 7f6c06a00640 1 --2- 172.21.15.143:0/3505019480 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6be806b310 0x7f6be806d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:59:59.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.899+0000 7f6c06a00640 1 -- 172.21.15.143:0/3505019480 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6c08089140 con 0x7f6c1013ea00 2024-01-16T14:59:59.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.899+0000 7f6c0ea00640 1 --2- 172.21.15.143:0/3505019480 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6be806b310 0x7f6be806d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:59:59.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.900+0000 7f6c0ea00640 1 --2- 172.21.15.143:0/3505019480 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6be806b310 0x7f6be806d7d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6c00009b40 tx=0x7f6c00009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:59:59.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T14:59:59.904+0000 7f6c06a00640 1 -- 172.21.15.143:0/3505019480 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6c08054390 con 0x7f6c1013ea00 2024-01-16T15:00:00.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.092+0000 7f6c15600640 1 -- 172.21.15.143:0/3505019480 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6c1004e8b0 con 0x7f6c1013ea00 2024-01-16T15:00:00.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.093+0000 7f6c06a00640 1 -- 172.21.15.143:0/3505019480 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6c08018020 con 0x7f6c1013ea00 2024-01-16T15:00:00.073 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-16T15:00:00.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.095+0000 7f6c15600640 1 -- 172.21.15.143:0/3505019480 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6be806b310 msgr2=0x7f6be806d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:00.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.095+0000 7f6c15600640 1 --2- 172.21.15.143:0/3505019480 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6be806b310 0x7f6be806d7d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6c00009b40 tx=0x7f6c00009190 comp rx=0 tx=0).stop 2024-01-16T15:00:00.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.095+0000 7f6c15600640 1 -- 172.21.15.143:0/3505019480 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c1013ea00 msgr2=0x7f6c10154480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:00.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.095+0000 7f6c15600640 1 --2- 172.21.15.143:0/3505019480 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c1013ea00 0x7f6c10154480 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6c0800c780 tx=0x7f6c0800cc40 comp rx=0 tx=0).stop 2024-01-16T15:00:00.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.096+0000 7f6c15600640 1 -- 172.21.15.143:0/3505019480 shutdown_connections 2024-01-16T15:00:00.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.096+0000 7f6c15600640 1 --2- 172.21.15.143:0/3505019480 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6be806b310 0x7f6be806d7d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:00.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.096+0000 7f6c15600640 1 --2- 172.21.15.143:0/3505019480 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c1013ea00 0x7f6c10154480 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:00.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.096+0000 7f6c15600640 1 --2- 172.21.15.143:0/3505019480 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c1013e3e0 0x7f6c100b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:00.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.096+0000 7f6c15600640 1 --2- 172.21.15.143:0/3505019480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c100cf320 0x7f6c100c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:00.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.096+0000 7f6c15600640 1 -- 172.21.15.143:0/3505019480 >> 172.21.15.143:0/3505019480 conn(0x7f6c100b2e40 msgr2=0x7f6c1015e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:00.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.096+0000 7f6c15600640 1 -- 172.21.15.143:0/3505019480 shutdown_connections 2024-01-16T15:00:00.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.096+0000 7f6c15600640 1 -- 172.21.15.143:0/3505019480 wait complete. 2024-01-16T15:00:00.094 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-16T15:00:00.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.314+0000 7f6900e00640 1 Processor -- start 2024-01-16T15:00:00.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.314+0000 7f6900e00640 1 -- start start 2024-01-16T15:00:00.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.316+0000 7f6900e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68fc13d120 0x7f68fc15af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:00.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.316+0000 7f6900e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f68fc06d6c0 0x7f68fc06db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:00.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.316+0000 7f6900e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f68fc0b0a70 0x7f68fc0b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:00.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.316+0000 7f6900e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68fc06e060 con 0x7f68fc06d6c0 2024-01-16T15:00:00.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.316+0000 7f6900e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68fc13f9e0 con 0x7f68fc0b0a70 2024-01-16T15:00:00.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.316+0000 7f6900e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc13fcc0 con 0x7f68fc13d120 2024-01-16T15:00:00.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.316+0000 7f68f2800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f68fc06d6c0 0x7f68fc06db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:00.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.316+0000 7f68f2800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f68fc06d6c0 0x7f68fc06db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57070/0 (socket says 172.21.15.143:57070) 2024-01-16T15:00:00.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.316+0000 7f68f2800640 1 -- 172.21.15.143:0/1605191795 learned_addr learned my addr 172.21.15.143:0/1605191795 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:00.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.317+0000 7f68fbe00640 1 --2- 172.21.15.143:0/1605191795 >> v2:172.21.15.204:3300/0 conn(0x7f68fc0b0a70 0x7f68fc0b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:00.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.317+0000 7f68fb400640 1 --2- 172.21.15.143:0/1605191795 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68fc13d120 0x7f68fc15af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:00.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.317+0000 7f68f2800640 1 -- 172.21.15.143:0/1605191795 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68fc13d120 msgr2=0x7f68fc15af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:00.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.317+0000 7f68f2800640 1 --2- 172.21.15.143:0/1605191795 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68fc13d120 0x7f68fc15af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:00.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.317+0000 7f68f2800640 1 -- 172.21.15.143:0/1605191795 >> v2:172.21.15.204:3300/0 conn(0x7f68fc0b0a70 msgr2=0x7f68fc0b0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:00.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.317+0000 7f68f2800640 1 --2- 172.21.15.143:0/1605191795 >> v2:172.21.15.204:3300/0 conn(0x7f68fc0b0a70 0x7f68fc0b0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:00.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.317+0000 7f68f2800640 1 -- 172.21.15.143:0/1605191795 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68fc13ffa0 con 0x7f68fc06d6c0 2024-01-16T15:00:00.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.318+0000 7f68f2800640 1 --2- 172.21.15.143:0/1605191795 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68fc06d6c0 0x7f68fc06db20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f68fc151660 tx=0x7f68e402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=949d743887fee5d1 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:00.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.318+0000 7f68faa00640 1 -- 172.21.15.143:0/1605191795 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e402c9e0 con 0x7f68fc06d6c0 2024-01-16T15:00:00.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.318+0000 7f68faa00640 1 -- 172.21.15.143:0/1605191795 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e402cb40 con 0x7f68fc06d6c0 2024-01-16T15:00:00.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.318+0000 7f68faa00640 1 -- 172.21.15.143:0/1605191795 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e40343f0 con 0x7f68fc06d6c0 2024-01-16T15:00:00.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.319+0000 7f6900e00640 1 -- 172.21.15.143:0/1605191795 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68fc06d6c0 msgr2=0x7f68fc06db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:00.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.319+0000 7f6900e00640 1 --2- 172.21.15.143:0/1605191795 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68fc06d6c0 0x7f68fc06db20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f68fc151660 tx=0x7f68e402be40 comp rx=0 tx=0).stop 2024-01-16T15:00:00.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.319+0000 7f6900e00640 1 -- 172.21.15.143:0/1605191795 shutdown_connections 2024-01-16T15:00:00.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.319+0000 7f6900e00640 1 --2- 172.21.15.143:0/1605191795 >> v2:172.21.15.204:3300/0 conn(0x7f68fc0b0a70 0x7f68fc0b0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:00.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.319+0000 7f6900e00640 1 --2- 172.21.15.143:0/1605191795 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68fc06d6c0 0x7f68fc06db20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:00.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.319+0000 7f6900e00640 1 --2- 172.21.15.143:0/1605191795 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68fc13d120 0x7f68fc15af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:00.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.319+0000 7f6900e00640 1 -- 172.21.15.143:0/1605191795 >> 172.21.15.143:0/1605191795 conn(0x7f68fc0cb830 msgr2=0x7f68fc0cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:00.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.319+0000 7f6900e00640 1 -- 172.21.15.143:0/1605191795 shutdown_connections 2024-01-16T15:00:00.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.319+0000 7f6900e00640 1 -- 172.21.15.143:0/1605191795 wait complete. 2024-01-16T15:00:00.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.321+0000 7f6900e00640 1 Processor -- start 2024-01-16T15:00:00.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.321+0000 7f6900e00640 1 -- start start 2024-01-16T15:00:00.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.322+0000 7f6900e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68fc06d6c0 0x7f68fc13b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:00.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.322+0000 7f6900e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68fc0b0a70 0x7f68fc14f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:00.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.322+0000 7f6900e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68fc13d120 0x7f68fc14fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:00.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.322+0000 7f6900e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc0b2f00 con 0x7f68fc0b0a70 2024-01-16T15:00:00.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.322+0000 7f6900e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc15c9a0 con 0x7f68fc06d6c0 2024-01-16T15:00:00.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.322+0000 7f6900e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc15d570 con 0x7f68fc13d120 2024-01-16T15:00:00.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.322+0000 7f68f2800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68fc0b0a70 0x7f68fc14f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:00.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.322+0000 7f68f2800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68fc0b0a70 0x7f68fc14f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57080/0 (socket says 172.21.15.143:57080) 2024-01-16T15:00:00.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.322+0000 7f68f2800640 1 -- 172.21.15.143:0/4058197582 learned_addr learned my addr 172.21.15.143:0/4058197582 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:00.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.323+0000 7f68fb400640 1 --2- 172.21.15.143:0/4058197582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68fc06d6c0 0x7f68fc13b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:00.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.323+0000 7f68fbe00640 1 --2- 172.21.15.143:0/4058197582 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68fc13d120 0x7f68fc14fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:00.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.323+0000 7f68f2800640 1 -- 172.21.15.143:0/4058197582 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68fc13d120 msgr2=0x7f68fc14fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:00.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.323+0000 7f68f2800640 1 --2- 172.21.15.143:0/4058197582 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68fc13d120 0x7f68fc14fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:00.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.323+0000 7f68f2800640 1 -- 172.21.15.143:0/4058197582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68fc06d6c0 msgr2=0x7f68fc13b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:00.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.323+0000 7f68f2800640 1 --2- 172.21.15.143:0/4058197582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68fc06d6c0 0x7f68fc13b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:00.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.323+0000 7f68f2800640 1 -- 172.21.15.143:0/4058197582 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e4009b30 con 0x7f68fc0b0a70 2024-01-16T15:00:00.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.323+0000 7f68fb400640 1 --2- 172.21.15.143:0/4058197582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68fc06d6c0 0x7f68fc13b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T15:00:00.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.323+0000 7f68f2800640 1 --2- 172.21.15.143:0/4058197582 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68fc0b0a70 0x7f68fc14f6a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f68e4000f80 tx=0x7f68e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:00.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.323+0000 7f68f3e00640 1 -- 172.21.15.143:0/4058197582 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e4009510 con 0x7f68fc0b0a70 2024-01-16T15:00:00.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.323+0000 7f68f3e00640 1 -- 172.21.15.143:0/4058197582 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e403e450 con 0x7f68fc0b0a70 2024-01-16T15:00:00.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.323+0000 7f68f3e00640 1 -- 172.21.15.143:0/4058197582 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e403e800 con 0x7f68fc0b0a70 2024-01-16T15:00:00.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.323+0000 7f6900e00640 1 -- 172.21.15.143:0/4058197582 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68fc15cc40 con 0x7f68fc0b0a70 2024-01-16T15:00:00.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.324+0000 7f6900e00640 1 -- 172.21.15.143:0/4058197582 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68fc15d200 con 0x7f68fc0b0a70 2024-01-16T15:00:00.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.325+0000 7f6900e00640 1 -- 172.21.15.143:0/4058197582 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68fc0cda80 con 0x7f68fc0b0a70 2024-01-16T15:00:00.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.326+0000 7f68f3e00640 1 -- 172.21.15.143:0/4058197582 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f68e403e960 con 0x7f68fc0b0a70 2024-01-16T15:00:00.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.326+0000 7f68f3e00640 1 --2- 172.21.15.143:0/4058197582 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f68d406b2f0 0x7f68d406d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:00.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.327+0000 7f68f3e00640 1 -- 172.21.15.143:0/4058197582 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f68e40ac640 con 0x7f68fc0b0a70 2024-01-16T15:00:00.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.327+0000 7f68fb400640 1 --2- 172.21.15.143:0/4058197582 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f68d406b2f0 0x7f68d406d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:00.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.327+0000 7f68fb400640 1 --2- 172.21.15.143:0/4058197582 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f68d406b2f0 0x7f68d406d7b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f68ec006fd0 tx=0x7f68ec008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:00.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.331+0000 7f68f3e00640 1 -- 172.21.15.143:0/4058197582 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f68fc0cda80 con 0x7f68fc0b0a70 2024-01-16T15:00:00.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.521+0000 7f6900e00640 1 -- 172.21.15.143:0/4058197582 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f68fc0cda80 con 0x7f68fc0b0a70 2024-01-16T15:00:00.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.522+0000 7f68f3e00640 1 -- 172.21.15.143:0/4058197582 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f68fc0cda80 con 0x7f68fc0b0a70 2024-01-16T15:00:00.502 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:00:00.502 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-16T15:00:00.503 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-16T15:00:00.503 INFO:teuthology.orchestra.run.smithi143.stdout: "poolnum": 1, 2024-01-16T15:00:00.503 INFO:teuthology.orchestra.run.smithi143.stdout: "poolname": ".mgr" 2024-01-16T15:00:00.503 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:00:00.503 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-16T15:00:00.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.524+0000 7f6900e00640 1 -- 172.21.15.143:0/4058197582 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f68d406b2f0 msgr2=0x7f68d406d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:00.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.524+0000 7f6900e00640 1 --2- 172.21.15.143:0/4058197582 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f68d406b2f0 0x7f68d406d7b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f68ec006fd0 tx=0x7f68ec008040 comp rx=0 tx=0).stop 2024-01-16T15:00:00.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.524+0000 7f6900e00640 1 -- 172.21.15.143:0/4058197582 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68fc0b0a70 msgr2=0x7f68fc14f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:00.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.524+0000 7f6900e00640 1 --2- 172.21.15.143:0/4058197582 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68fc0b0a70 0x7f68fc14f6a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f68e4000f80 tx=0x7f68e4004a40 comp rx=0 tx=0).stop 2024-01-16T15:00:00.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.525+0000 7f6900e00640 1 -- 172.21.15.143:0/4058197582 shutdown_connections 2024-01-16T15:00:00.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.525+0000 7f6900e00640 1 --2- 172.21.15.143:0/4058197582 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f68d406b2f0 0x7f68d406d7b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:00.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.525+0000 7f6900e00640 1 --2- 172.21.15.143:0/4058197582 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68fc13d120 0x7f68fc14fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:00.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.525+0000 7f6900e00640 1 --2- 172.21.15.143:0/4058197582 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68fc0b0a70 0x7f68fc14f6a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:00.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.525+0000 7f6900e00640 1 --2- 172.21.15.143:0/4058197582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68fc06d6c0 0x7f68fc13b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:00.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.525+0000 7f6900e00640 1 -- 172.21.15.143:0/4058197582 >> 172.21.15.143:0/4058197582 conn(0x7f68fc0cb830 msgr2=0x7f68fc0af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:00.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.525+0000 7f6900e00640 1 -- 172.21.15.143:0/4058197582 shutdown_connections 2024-01-16T15:00:00.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.525+0000 7f6900e00640 1 -- 172.21.15.143:0/4058197582 wait complete. 2024-01-16T15:00:00.523 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-16T15:00:00.523 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-16T15:00:00.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.754+0000 7f42cb400640 1 Processor -- start 2024-01-16T15:00:00.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.754+0000 7f42cb400640 1 -- start start 2024-01-16T15:00:00.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.755+0000 7f42cb400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f42c40b2250 0x7f42c415df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:00.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.755+0000 7f42cb400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42c40b2630 0x7f42c40ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:00.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.755+0000 7f42cb400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42c415ab80 con 0x7f42c40cd470 2024-01-16T15:00:00.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.756+0000 7f42cb400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42c415a990 con 0x7f42c40b2250 2024-01-16T15:00:00.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.756+0000 7f42cb400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42c415e680 con 0x7f42c40b2630 2024-01-16T15:00:00.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.756+0000 7f42c8a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f42c40b2250 0x7f42c415df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:00.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.756+0000 7f42c9400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f42c40cd470 0x7f42c415b080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55034/0 (socket says 172.21.15.143:55034) 2024-01-16T15:00:00.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.756+0000 7f42c9400640 1 -- 172.21.15.143:0/271965193 learned_addr learned my addr 172.21.15.143:0/271965193 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:00.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.756+0000 7f42bbc00640 1 --2- 172.21.15.143:0/271965193 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42c40b2630 0x7f42c40ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:00.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.757+0000 7f42c8a00640 1 -- 172.21.15.143:0/271965193 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42c40b2630 msgr2=0x7f42c40ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:00.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.757+0000 7f42c8a00640 1 --2- 172.21.15.143:0/271965193 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42c40b2630 0x7f42c40ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:00.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.757+0000 7f42c8a00640 1 -- 172.21.15.143:0/271965193 >> v1:172.21.15.143:6789/0 conn(0x7f42c40cd470 legacy=0x7f42c415b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:00.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.757+0000 7f42c8a00640 1 -- 172.21.15.143:0/271965193 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c415e960 con 0x7f42c40b2250 2024-01-16T15:00:00.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.757+0000 7f42bbc00640 1 --2- 172.21.15.143:0/271965193 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42c40b2630 0x7f42c40ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T15:00:00.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.758+0000 7f42c8a00640 1 --2- 172.21.15.143:0/271965193 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42c40b2250 0x7f42c415df80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f42b40097a0 tx=0x7f42b402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f14a2004addd9a6 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:00.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.758+0000 7f42c3e00640 1 -- 172.21.15.143:0/271965193 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b4031830 con 0x7f42c40b2250 2024-01-16T15:00:00.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.758+0000 7f42c3e00640 1 -- 172.21.15.143:0/271965193 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42b4031990 con 0x7f42c40b2250 2024-01-16T15:00:00.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.758+0000 7f42c3e00640 1 -- 172.21.15.143:0/271965193 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b4031d20 con 0x7f42c40b2250 2024-01-16T15:00:00.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.759+0000 7f42cb400640 1 -- 172.21.15.143:0/271965193 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42c40b2250 msgr2=0x7f42c415df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:00.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.759+0000 7f42cb400640 1 --2- 172.21.15.143:0/271965193 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42c40b2250 0x7f42c415df80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f42b40097a0 tx=0x7f42b402b8d0 comp rx=0 tx=0).stop 2024-01-16T15:00:00.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.759+0000 7f42cb400640 1 -- 172.21.15.143:0/271965193 shutdown_connections 2024-01-16T15:00:00.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.759+0000 7f42cb400640 1 --2- 172.21.15.143:0/271965193 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42c40b2630 0x7f42c40ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:00.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.759+0000 7f42cb400640 1 --2- 172.21.15.143:0/271965193 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42c40b2250 0x7f42c415df80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:00.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.759+0000 7f42cb400640 1 -- 172.21.15.143:0/271965193 >> 172.21.15.143:0/271965193 conn(0x7f42c406a920 msgr2=0x7f42c406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:00.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.759+0000 7f42cb400640 1 -- 172.21.15.143:0/271965193 shutdown_connections 2024-01-16T15:00:00.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.759+0000 7f42cb400640 1 -- 172.21.15.143:0/271965193 wait complete. 2024-01-16T15:00:00.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.761+0000 7f42cb400640 1 Processor -- start 2024-01-16T15:00:00.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.761+0000 7f42cb400640 1 -- start start 2024-01-16T15:00:00.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.762+0000 7f42cb400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42c40b2250 0x7f42c4154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:00.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.762+0000 7f42cb400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42c40b2630 0x7f42c4158820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:00.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.762+0000 7f42cb400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42c40cd470 0x7f42c4148f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:00.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.762+0000 7f42cb400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c4154ff0 con 0x7f42c40b2250 2024-01-16T15:00:00.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.762+0000 7f42cb400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c4153a10 con 0x7f42c40cd470 2024-01-16T15:00:00.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.762+0000 7f42cb400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42c4153cf0 con 0x7f42c40b2630 2024-01-16T15:00:00.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.762+0000 7f42c8a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42c40b2250 0x7f42c4154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:00.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.762+0000 7f42c8a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42c40b2250 0x7f42c4154ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57086/0 (socket says 172.21.15.143:57086) 2024-01-16T15:00:00.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.762+0000 7f42c8a00640 1 -- 172.21.15.143:0/1560246029 learned_addr learned my addr 172.21.15.143:0/1560246029 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:00.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.763+0000 7f42bbc00640 1 --2- 172.21.15.143:0/1560246029 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42c40b2630 0x7f42c4158820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:00.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.763+0000 7f42c9400640 1 --2- 172.21.15.143:0/1560246029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42c40cd470 0x7f42c4148f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:00.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.763+0000 7f42c8a00640 1 -- 172.21.15.143:0/1560246029 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42c40b2630 msgr2=0x7f42c4158820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:00.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.763+0000 7f42c8a00640 1 --2- 172.21.15.143:0/1560246029 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42c40b2630 0x7f42c4158820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:00.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.763+0000 7f42c8a00640 1 -- 172.21.15.143:0/1560246029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42c40cd470 msgr2=0x7f42c4148f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:00.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.763+0000 7f42c8a00640 1 --2- 172.21.15.143:0/1560246029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42c40cd470 0x7f42c4148f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:00.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.763+0000 7f42c8a00640 1 -- 172.21.15.143:0/1560246029 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b4009400 con 0x7f42c40b2250 2024-01-16T15:00:00.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.763+0000 7f42bbc00640 1 --2- 172.21.15.143:0/1560246029 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42c40b2630 0x7f42c4158820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:00.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.763+0000 7f42c9400640 1 --2- 172.21.15.143:0/1560246029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42c40cd470 0x7f42c4148f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:00.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.763+0000 7f42c8a00640 1 --2- 172.21.15.143:0/1560246029 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42c40b2250 0x7f42c4154ab0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f42b402bee0 tx=0x7f42b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:00.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.763+0000 7f42c1600640 1 -- 172.21.15.143:0/1560246029 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b403a6f0 con 0x7f42c40b2250 2024-01-16T15:00:00.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.763+0000 7f42c1600640 1 -- 172.21.15.143:0/1560246029 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42b403a870 con 0x7f42c40b2250 2024-01-16T15:00:00.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.763+0000 7f42cb400640 1 -- 172.21.15.143:0/1560246029 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42c4153fd0 con 0x7f42c40b2250 2024-01-16T15:00:00.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.763+0000 7f42c1600640 1 -- 172.21.15.143:0/1560246029 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b403ac20 con 0x7f42c40b2250 2024-01-16T15:00:00.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.764+0000 7f42cb400640 1 -- 172.21.15.143:0/1560246029 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42c4154590 con 0x7f42c40b2250 2024-01-16T15:00:00.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.764+0000 7f42cb400640 1 -- 172.21.15.143:0/1560246029 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42c4154780 con 0x7f42c40b2250 2024-01-16T15:00:00.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.765+0000 7f42c1600640 1 -- 172.21.15.143:0/1560246029 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f42b404c3b0 con 0x7f42c40b2250 2024-01-16T15:00:00.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.766+0000 7f42c1600640 1 --2- 172.21.15.143:0/1560246029 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f429c06b2c0 0x7f429c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:00.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.766+0000 7f42c1600640 1 -- 172.21.15.143:0/1560246029 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f42b40ab5b0 con 0x7f42c40b2250 2024-01-16T15:00:00.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.767+0000 7f42bbc00640 1 --2- 172.21.15.143:0/1560246029 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f429c06b2c0 0x7f429c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:00.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.767+0000 7f42bbc00640 1 --2- 172.21.15.143:0/1560246029 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f429c06b2c0 0x7f429c06d780 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f42ac006fd0 tx=0x7f42ac008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:00.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.771+0000 7f42c1600640 1 -- 172.21.15.143:0/1560246029 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f42b4076800 con 0x7f42c40b2250 2024-01-16T15:00:00.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:00.962+0000 7f42cb400640 1 -- 172.21.15.143:0/1560246029 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f42c4067470 con 0x7f42c40b2250 2024-01-16T15:00:01.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.578+0000 7f42c1600640 1 -- 172.21.15.143:0/1560246029 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f42b4079e20 con 0x7f42c40b2250 2024-01-16T15:00:01.559 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' created 2024-01-16T15:00:01.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.582+0000 7f42cb400640 1 -- 172.21.15.143:0/1560246029 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f429c06b2c0 msgr2=0x7f429c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:01.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.582+0000 7f42cb400640 1 --2- 172.21.15.143:0/1560246029 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f429c06b2c0 0x7f429c06d780 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f42ac006fd0 tx=0x7f42ac008040 comp rx=0 tx=0).stop 2024-01-16T15:00:01.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.582+0000 7f42cb400640 1 -- 172.21.15.143:0/1560246029 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42c40b2250 msgr2=0x7f42c4154ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:01.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.582+0000 7f42cb400640 1 --2- 172.21.15.143:0/1560246029 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42c40b2250 0x7f42c4154ab0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f42b402bee0 tx=0x7f42b40046c0 comp rx=0 tx=0).stop 2024-01-16T15:00:01.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.582+0000 7f42cb400640 1 -- 172.21.15.143:0/1560246029 shutdown_connections 2024-01-16T15:00:01.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.582+0000 7f42cb400640 1 --2- 172.21.15.143:0/1560246029 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f429c06b2c0 0x7f429c06d780 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:01.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.582+0000 7f42cb400640 1 --2- 172.21.15.143:0/1560246029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42c40cd470 0x7f42c4148f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:01.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.582+0000 7f42cb400640 1 --2- 172.21.15.143:0/1560246029 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42c40b2630 0x7f42c4158820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:01.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.582+0000 7f42cb400640 1 --2- 172.21.15.143:0/1560246029 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42c40b2250 0x7f42c4154ab0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:01.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.582+0000 7f42cb400640 1 -- 172.21.15.143:0/1560246029 >> 172.21.15.143:0/1560246029 conn(0x7f42c406a920 msgr2=0x7f42c40adfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:01.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.583+0000 7f42cb400640 1 -- 172.21.15.143:0/1560246029 shutdown_connections 2024-01-16T15:00:01.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.583+0000 7f42cb400640 1 -- 172.21.15.143:0/1560246029 wait complete. 2024-01-16T15:00:01.586 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-16T15:00:01.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.801+0000 7f535b000640 1 Processor -- start 2024-01-16T15:00:01.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.801+0000 7f535b000640 1 -- start start 2024-01-16T15:00:01.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.802+0000 7f535b000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f535406de10 0x7f535415e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:01.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.802+0000 7f535b000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53540c49f0 0x7f53540705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:01.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.802+0000 7f535b000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f535415ec90 con 0x7f5354072160 2024-01-16T15:00:01.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.802+0000 7f535b000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53540cd530 con 0x7f535406de10 2024-01-16T15:00:01.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.802+0000 7f535b000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53540c4ed0 con 0x7f53540c49f0 2024-01-16T15:00:01.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.802+0000 7f5353e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f5354072160 0x7f535406d700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55042/0 (socket says 172.21.15.143:55042) 2024-01-16T15:00:01.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.802+0000 7f5353e00640 1 -- 172.21.15.143:0/1545869492 learned_addr learned my addr 172.21.15.143:0/1545869492 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:01.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.803+0000 7f534be00640 1 --2- 172.21.15.143:0/1545869492 >> v2:172.21.15.143:3300/0 conn(0x7f535406de10 0x7f535415e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:01.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.803+0000 7f5359000640 1 --2- 172.21.15.143:0/1545869492 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53540c49f0 0x7f53540705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:01.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.803+0000 7f5353400640 1 -- 172.21.15.143:0/1545869492 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2012803757 0 0) 0x7f5344001110 con 0x7f5354072160 2024-01-16T15:00:01.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.803+0000 7f5353400640 1 -- 172.21.15.143:0/1545869492 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5334003520 con 0x7f5354072160 2024-01-16T15:00:01.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.803+0000 7f534be00640 1 -- 172.21.15.143:0/1545869492 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53540c49f0 msgr2=0x7f53540705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:01.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.803+0000 7f534be00640 1 --2- 172.21.15.143:0/1545869492 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53540c49f0 0x7f53540705c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:01.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.803+0000 7f534be00640 1 -- 172.21.15.143:0/1545869492 >> v1:172.21.15.143:6789/0 conn(0x7f5354072160 legacy=0x7f535406d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:01.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.803+0000 7f534be00640 1 -- 172.21.15.143:0/1545869492 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53540c51b0 con 0x7f535406de10 2024-01-16T15:00:01.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.803+0000 7f5359000640 1 --2- 172.21.15.143:0/1545869492 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53540c49f0 0x7f53540705c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:01.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.804+0000 7f534be00640 1 --2- 172.21.15.143:0/1545869492 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f535406de10 0x7f535415e750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f533c0097a0 tx=0x7f533c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2fe261f4afbe866 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:01.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.804+0000 7f5353400640 1 -- 172.21.15.143:0/1545869492 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533c031830 con 0x7f535406de10 2024-01-16T15:00:01.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.804+0000 7f5353400640 1 -- 172.21.15.143:0/1545869492 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f533c031990 con 0x7f535406de10 2024-01-16T15:00:01.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.804+0000 7f5353400640 1 -- 172.21.15.143:0/1545869492 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533c031d20 con 0x7f535406de10 2024-01-16T15:00:01.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.805+0000 7f535b000640 1 -- 172.21.15.143:0/1545869492 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f535406de10 msgr2=0x7f535415e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:01.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.805+0000 7f535b000640 1 --2- 172.21.15.143:0/1545869492 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f535406de10 0x7f535415e750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f533c0097a0 tx=0x7f533c02b8d0 comp rx=0 tx=0).stop 2024-01-16T15:00:01.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.805+0000 7f535b000640 1 -- 172.21.15.143:0/1545869492 shutdown_connections 2024-01-16T15:00:01.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.805+0000 7f535b000640 1 --2- 172.21.15.143:0/1545869492 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53540c49f0 0x7f53540705c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:01.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.805+0000 7f535b000640 1 --2- 172.21.15.143:0/1545869492 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f535406de10 0x7f535415e750 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:01.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.805+0000 7f535b000640 1 -- 172.21.15.143:0/1545869492 >> 172.21.15.143:0/1545869492 conn(0x7f53540b0c50 msgr2=0x7f53540b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:01.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.805+0000 7f535b000640 1 -- 172.21.15.143:0/1545869492 shutdown_connections 2024-01-16T15:00:01.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.805+0000 7f535b000640 1 -- 172.21.15.143:0/1545869492 wait complete. 2024-01-16T15:00:01.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.807+0000 7f535b000640 1 Processor -- start 2024-01-16T15:00:01.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.807+0000 7f535b000640 1 -- start start 2024-01-16T15:00:01.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.808+0000 7f535b000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f535406de10 0x7f5354148df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:01.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.808+0000 7f535b000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5354072160 0x7f53540999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:01.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.808+0000 7f535b000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53540c49f0 0x7f5354099f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:01.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.808+0000 7f535b000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535409b5c0 con 0x7f535406de10 2024-01-16T15:00:01.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.808+0000 7f535b000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535415ce50 con 0x7f53540c49f0 2024-01-16T15:00:01.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.808+0000 7f535b000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f535409a520 con 0x7f5354072160 2024-01-16T15:00:01.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.808+0000 7f5353e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f535406de10 0x7f5354148df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:01.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.808+0000 7f5353e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f535406de10 0x7f5354148df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57098/0 (socket says 172.21.15.143:57098) 2024-01-16T15:00:01.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.808+0000 7f5353e00640 1 -- 172.21.15.143:0/86973963 learned_addr learned my addr 172.21.15.143:0/86973963 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:01.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.808+0000 7f5359000640 1 --2- 172.21.15.143:0/86973963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53540c49f0 0x7f5354099f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:01.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.808+0000 7f534be00640 1 --2- 172.21.15.143:0/86973963 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5354072160 0x7f53540999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:01.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.808+0000 7f5353e00640 1 -- 172.21.15.143:0/86973963 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5354072160 msgr2=0x7f53540999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:01.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.808+0000 7f5353e00640 1 --2- 172.21.15.143:0/86973963 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5354072160 0x7f53540999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:01.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.808+0000 7f5353e00640 1 -- 172.21.15.143:0/86973963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53540c49f0 msgr2=0x7f5354099f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:01.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.808+0000 7f5353e00640 1 --2- 172.21.15.143:0/86973963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53540c49f0 0x7f5354099f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:01.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.808+0000 7f5353e00640 1 -- 172.21.15.143:0/86973963 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f533c009400 con 0x7f535406de10 2024-01-16T15:00:01.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.809+0000 7f5353e00640 1 --2- 172.21.15.143:0/86973963 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f535406de10 0x7f5354148df0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f534400c700 tx=0x7f534400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:01.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.809+0000 7f5350c00640 1 -- 172.21.15.143:0/86973963 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5344005ce0 con 0x7f535406de10 2024-01-16T15:00:01.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.809+0000 7f5350c00640 1 -- 172.21.15.143:0/86973963 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5344005e40 con 0x7f535406de10 2024-01-16T15:00:01.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.809+0000 7f5350c00640 1 -- 172.21.15.143:0/86973963 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5344007290 con 0x7f535406de10 2024-01-16T15:00:01.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.809+0000 7f535b000640 1 -- 172.21.15.143:0/86973963 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f535409a800 con 0x7f535406de10 2024-01-16T15:00:01.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.809+0000 7f535b000640 1 -- 172.21.15.143:0/86973963 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f535409adc0 con 0x7f535406de10 2024-01-16T15:00:01.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.810+0000 7f535b000640 1 -- 172.21.15.143:0/86973963 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f535404e8b0 con 0x7f535406de10 2024-01-16T15:00:01.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.811+0000 7f5350c00640 1 -- 172.21.15.143:0/86973963 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f53440074b0 con 0x7f535406de10 2024-01-16T15:00:01.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.812+0000 7f5350c00640 1 --2- 172.21.15.143:0/86973963 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f532c06b2c0 0x7f532c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:01.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.812+0000 7f5350c00640 1 -- 172.21.15.143:0/86973963 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f5344088f20 con 0x7f535406de10 2024-01-16T15:00:01.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.812+0000 7f534be00640 1 --2- 172.21.15.143:0/86973963 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f532c06b2c0 0x7f532c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:01.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.813+0000 7f534be00640 1 --2- 172.21.15.143:0/86973963 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f532c06b2c0 0x7f532c06d780 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f533c036000 tx=0x7f533c03c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:01.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.817+0000 7f5350c00640 1 -- 172.21.15.143:0/86973963 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5344057960 con 0x7f535406de10 2024-01-16T15:00:01.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:01.993+0000 7f535b000640 1 -- 172.21.15.143:0/86973963 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f535415ec90 con 0x7f535406de10 2024-01-16T15:00:02.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.593+0000 7f5350c00640 1 -- 172.21.15.143:0/86973963 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7f535415ec90 con 0x7f535406de10 2024-01-16T15:00:02.573 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' created 2024-01-16T15:00:02.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.596+0000 7f535b000640 1 -- 172.21.15.143:0/86973963 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f532c06b2c0 msgr2=0x7f532c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:02.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.596+0000 7f535b000640 1 --2- 172.21.15.143:0/86973963 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f532c06b2c0 0x7f532c06d780 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f533c036000 tx=0x7f533c03c040 comp rx=0 tx=0).stop 2024-01-16T15:00:02.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.596+0000 7f535b000640 1 -- 172.21.15.143:0/86973963 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f535406de10 msgr2=0x7f5354148df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:02.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.596+0000 7f535b000640 1 --2- 172.21.15.143:0/86973963 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f535406de10 0x7f5354148df0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f534400c700 tx=0x7f534400cb00 comp rx=0 tx=0).stop 2024-01-16T15:00:02.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.596+0000 7f535b000640 1 -- 172.21.15.143:0/86973963 shutdown_connections 2024-01-16T15:00:02.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.596+0000 7f535b000640 1 --2- 172.21.15.143:0/86973963 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f532c06b2c0 0x7f532c06d780 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:02.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.596+0000 7f535b000640 1 --2- 172.21.15.143:0/86973963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53540c49f0 0x7f5354099f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:02.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.596+0000 7f535b000640 1 --2- 172.21.15.143:0/86973963 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5354072160 0x7f53540999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:02.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.596+0000 7f535b000640 1 --2- 172.21.15.143:0/86973963 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f535406de10 0x7f5354148df0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:02.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.596+0000 7f535b000640 1 -- 172.21.15.143:0/86973963 >> 172.21.15.143:0/86973963 conn(0x7f53540b0c50 msgr2=0x7f53540b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:02.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.597+0000 7f535b000640 1 -- 172.21.15.143:0/86973963 shutdown_connections 2024-01-16T15:00:02.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.597+0000 7f535b000640 1 -- 172.21.15.143:0/86973963 wait complete. 2024-01-16T15:00:02.613 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-16T15:00:02.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.815+0000 7fe371e00640 1 Processor -- start 2024-01-16T15:00:02.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.816+0000 7fe371e00640 1 -- start start 2024-01-16T15:00:02.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.817+0000 7fe371e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe36c072160 0x7fe36c15e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:02.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.817+0000 7fe371e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe36c15ec90 con 0x7fe36c06de10 2024-01-16T15:00:02.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.817+0000 7fe371e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe36c0cd530 con 0x7fe36c0c49f0 2024-01-16T15:00:02.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.817+0000 7fe371e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe36c0c4ed0 con 0x7fe36c072160 2024-01-16T15:00:02.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.817+0000 7fe36b400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe36c06de10 0x7fe36c0705c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55058/0 (socket says 172.21.15.143:55058) 2024-01-16T15:00:02.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.817+0000 7fe36aa00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe36c072160 0x7fe36c15e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:02.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.817+0000 7fe36b400640 1 -- 172.21.15.143:0/3392376733 learned_addr learned my addr 172.21.15.143:0/3392376733 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:02.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.818+0000 7fe36a000640 1 -- 172.21.15.143:0/3392376733 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1772005459 0 0) 0x7fe364001110 con 0x7fe36c06de10 2024-01-16T15:00:02.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.818+0000 7fe36a000640 1 -- 172.21.15.143:0/3392376733 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe34c005730 con 0x7fe36c06de10 2024-01-16T15:00:02.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.818+0000 7fe36aa00640 1 -- 172.21.15.143:0/3392376733 >> v1:172.21.15.204:6789/0 conn(0x7fe36c0c49f0 legacy=0x7fe36c06d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:02.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.818+0000 7fe36aa00640 1 -- 172.21.15.143:0/3392376733 >> v1:172.21.15.143:6789/0 conn(0x7fe36c06de10 legacy=0x7fe36c0705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:02.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.818+0000 7fe36aa00640 1 -- 172.21.15.143:0/3392376733 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36c0c51b0 con 0x7fe36c072160 2024-01-16T15:00:02.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.819+0000 7fe36aa00640 1 --2- 172.21.15.143:0/3392376733 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe36c072160 0x7fe36c15e750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe35c0097a0 tx=0x7fe35c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a3daf4906b2ea8 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:02.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.820+0000 7fe36a000640 1 -- 172.21.15.143:0/3392376733 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c031830 con 0x7fe36c072160 2024-01-16T15:00:02.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.820+0000 7fe36a000640 1 -- 172.21.15.143:0/3392376733 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35c031990 con 0x7fe36c072160 2024-01-16T15:00:02.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.820+0000 7fe36a000640 1 -- 172.21.15.143:0/3392376733 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c031d20 con 0x7fe36c072160 2024-01-16T15:00:02.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.820+0000 7fe371e00640 1 -- 172.21.15.143:0/3392376733 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe36c072160 msgr2=0x7fe36c15e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:02.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.820+0000 7fe371e00640 1 --2- 172.21.15.143:0/3392376733 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe36c072160 0x7fe36c15e750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe35c0097a0 tx=0x7fe35c02b8d0 comp rx=0 tx=0).stop 2024-01-16T15:00:02.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.820+0000 7fe371e00640 1 -- 172.21.15.143:0/3392376733 shutdown_connections 2024-01-16T15:00:02.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.820+0000 7fe371e00640 1 --2- 172.21.15.143:0/3392376733 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe36c072160 0x7fe36c15e750 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:02.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.820+0000 7fe371e00640 1 -- 172.21.15.143:0/3392376733 >> 172.21.15.143:0/3392376733 conn(0x7fe36c0b0c50 msgr2=0x7fe36c0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:02.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.821+0000 7fe371e00640 1 -- 172.21.15.143:0/3392376733 shutdown_connections 2024-01-16T15:00:02.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.821+0000 7fe371e00640 1 -- 172.21.15.143:0/3392376733 wait complete. 2024-01-16T15:00:02.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.823+0000 7fe371e00640 1 Processor -- start 2024-01-16T15:00:02.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.823+0000 7fe371e00640 1 -- start start 2024-01-16T15:00:02.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.824+0000 7fe371e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe36c06de10 0x7fe36c0cd930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:02.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.824+0000 7fe371e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe36c072160 0x7fe36c15c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:02.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.824+0000 7fe371e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe36c0c49f0 0x7fe36c157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:02.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.824+0000 7fe371e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c1ef240 con 0x7fe36c0c49f0 2024-01-16T15:00:02.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.824+0000 7fe371e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c1ef520 con 0x7fe36c06de10 2024-01-16T15:00:02.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.824+0000 7fe371e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c1ef800 con 0x7fe36c072160 2024-01-16T15:00:02.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.824+0000 7fe36b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe36c0c49f0 0x7fe36c157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:02.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.824+0000 7fe36b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe36c0c49f0 0x7fe36c157be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57122/0 (socket says 172.21.15.143:57122) 2024-01-16T15:00:02.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.824+0000 7fe36b400640 1 -- 172.21.15.143:0/3842591246 learned_addr learned my addr 172.21.15.143:0/3842591246 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:02.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.825+0000 7fe361e00640 1 --2- 172.21.15.143:0/3842591246 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe36c072160 0x7fe36c15c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:02.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.825+0000 7fe36aa00640 1 --2- 172.21.15.143:0/3842591246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe36c06de10 0x7fe36c0cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:02.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.825+0000 7fe36b400640 1 -- 172.21.15.143:0/3842591246 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe36c072160 msgr2=0x7fe36c15c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:02.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.825+0000 7fe36b400640 1 --2- 172.21.15.143:0/3842591246 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe36c072160 0x7fe36c15c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:02.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.825+0000 7fe36b400640 1 -- 172.21.15.143:0/3842591246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe36c06de10 msgr2=0x7fe36c0cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:02.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.825+0000 7fe36b400640 1 --2- 172.21.15.143:0/3842591246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe36c06de10 0x7fe36c0cd930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:02.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.825+0000 7fe36b400640 1 -- 172.21.15.143:0/3842591246 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35c009400 con 0x7fe36c0c49f0 2024-01-16T15:00:02.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.825+0000 7fe36b400640 1 --2- 172.21.15.143:0/3842591246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe36c0c49f0 0x7fe36c157be0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe36400c730 tx=0x7fe36400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:02.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.825+0000 7fe363e00640 1 -- 172.21.15.143:0/3842591246 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe364005ab0 con 0x7fe36c0c49f0 2024-01-16T15:00:02.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.825+0000 7fe363e00640 1 -- 172.21.15.143:0/3842591246 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe364005c10 con 0x7fe36c0c49f0 2024-01-16T15:00:02.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.825+0000 7fe371e00640 1 -- 172.21.15.143:0/3842591246 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe36c1efae0 con 0x7fe36c0c49f0 2024-01-16T15:00:02.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.825+0000 7fe363e00640 1 -- 172.21.15.143:0/3842591246 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe364001d30 con 0x7fe36c0c49f0 2024-01-16T15:00:02.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.826+0000 7fe371e00640 1 -- 172.21.15.143:0/3842591246 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36c0999d0 con 0x7fe36c0c49f0 2024-01-16T15:00:02.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.827+0000 7fe371e00640 1 -- 172.21.15.143:0/3842591246 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe36c1f0000 con 0x7fe36c0c49f0 2024-01-16T15:00:02.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.828+0000 7fe363e00640 1 -- 172.21.15.143:0/3842591246 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fe364007070 con 0x7fe36c0c49f0 2024-01-16T15:00:02.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.829+0000 7fe363e00640 1 --2- 172.21.15.143:0/3842591246 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe34406b2c0 0x7fe34406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:02.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.829+0000 7fe363e00640 1 -- 172.21.15.143:0/3842591246 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fe364088f20 con 0x7fe36c0c49f0 2024-01-16T15:00:02.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.829+0000 7fe36aa00640 1 --2- 172.21.15.143:0/3842591246 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe34406b2c0 0x7fe34406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:02.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.830+0000 7fe36aa00640 1 --2- 172.21.15.143:0/3842591246 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe34406b2c0 0x7fe34406d780 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe35c036000 tx=0x7fe35c008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:02.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:02.834+0000 7fe363e00640 1 -- 172.21.15.143:0/3842591246 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe36c1f0000 con 0x7fe36c0c49f0 2024-01-16T15:00:03.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.048+0000 7fe371e00640 1 -- 172.21.15.143:0/3842591246 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe36c15ec90 con 0x7fe36c0c49f0 2024-01-16T15:00:03.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.595+0000 7fe363e00640 1 -- 172.21.15.143:0/3842591246 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fe36c15ec90 con 0x7fe36c0c49f0 2024-01-16T15:00:03.575 INFO:teuthology.orchestra.run.smithi143.stderr:new fs with metadata pool 2 and data pool 3 2024-01-16T15:00:03.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.599+0000 7fe371e00640 1 -- 172.21.15.143:0/3842591246 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe34406b2c0 msgr2=0x7fe34406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:03.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.599+0000 7fe371e00640 1 --2- 172.21.15.143:0/3842591246 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe34406b2c0 0x7fe34406d780 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe35c036000 tx=0x7fe35c008030 comp rx=0 tx=0).stop 2024-01-16T15:00:03.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.599+0000 7fe371e00640 1 -- 172.21.15.143:0/3842591246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe36c0c49f0 msgr2=0x7fe36c157be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:03.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.599+0000 7fe371e00640 1 --2- 172.21.15.143:0/3842591246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe36c0c49f0 0x7fe36c157be0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe36400c730 tx=0x7fe36400cbf0 comp rx=0 tx=0).stop 2024-01-16T15:00:03.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.599+0000 7fe371e00640 1 -- 172.21.15.143:0/3842591246 shutdown_connections 2024-01-16T15:00:03.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.599+0000 7fe371e00640 1 --2- 172.21.15.143:0/3842591246 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe34406b2c0 0x7fe34406d780 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:03.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.599+0000 7fe371e00640 1 --2- 172.21.15.143:0/3842591246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe36c0c49f0 0x7fe36c157be0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:03.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.599+0000 7fe371e00640 1 --2- 172.21.15.143:0/3842591246 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe36c072160 0x7fe36c15c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:03.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.599+0000 7fe371e00640 1 --2- 172.21.15.143:0/3842591246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe36c06de10 0x7fe36c0cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:03.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.599+0000 7fe371e00640 1 -- 172.21.15.143:0/3842591246 >> 172.21.15.143:0/3842591246 conn(0x7fe36c0b0c50 msgr2=0x7fe36c0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:03.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.599+0000 7fe371e00640 1 -- 172.21.15.143:0/3842591246 shutdown_connections 2024-01-16T15:00:03.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.599+0000 7fe371e00640 1 -- 172.21.15.143:0/3842591246 wait complete. 2024-01-16T15:00:03.605 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T15:00:03.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.793+0000 7fa963e00640 1 Processor -- start 2024-01-16T15:00:03.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.793+0000 7fa963e00640 1 -- start start 2024-01-16T15:00:03.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.794+0000 7fa963e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa95c0b2250 0x7fa95c15df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:03.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.794+0000 7fa963e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa95c0b2630 0x7fa95c0ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:03.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.794+0000 7fa963e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa95c0cd470 0x7fa95c15b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:03.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.794+0000 7fa963e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa95c15a990 con 0x7fa95c0cd470 2024-01-16T15:00:03.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.794+0000 7fa963e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa95c15ab80 con 0x7fa95c0b2630 2024-01-16T15:00:03.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.794+0000 7fa963e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c15e680 con 0x7fa95c0b2250 2024-01-16T15:00:03.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.794+0000 7fa961e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa95c0cd470 0x7fa95c15b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:03.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.794+0000 7fa961e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa95c0cd470 0x7fa95c15b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57130/0 (socket says 172.21.15.143:57130) 2024-01-16T15:00:03.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.794+0000 7fa961e00640 1 -- 172.21.15.143:0/447924631 learned_addr learned my addr 172.21.15.143:0/447924631 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:03.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.795+0000 7fa960a00640 1 --2- 172.21.15.143:0/447924631 >> v2:172.21.15.204:3300/0 conn(0x7fa95c0b2630 0x7fa95c0ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:03.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.795+0000 7fa961400640 1 --2- 172.21.15.143:0/447924631 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa95c0b2250 0x7fa95c15df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:03.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.795+0000 7fa961e00640 1 -- 172.21.15.143:0/447924631 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa95c0b2250 msgr2=0x7fa95c15df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:03.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.795+0000 7fa961e00640 1 --2- 172.21.15.143:0/447924631 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa95c0b2250 0x7fa95c15df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:03.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.795+0000 7fa961e00640 1 -- 172.21.15.143:0/447924631 >> v2:172.21.15.204:3300/0 conn(0x7fa95c0b2630 msgr2=0x7fa95c0ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:03.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.795+0000 7fa961e00640 1 --2- 172.21.15.143:0/447924631 >> v2:172.21.15.204:3300/0 conn(0x7fa95c0b2630 0x7fa95c0ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:03.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.795+0000 7fa961e00640 1 -- 172.21.15.143:0/447924631 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa95c15e960 con 0x7fa95c0cd470 2024-01-16T15:00:03.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.795+0000 7fa961400640 1 --2- 172.21.15.143:0/447924631 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa95c0b2250 0x7fa95c15df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:03.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.796+0000 7fa961e00640 1 --2- 172.21.15.143:0/447924631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa95c0cd470 0x7fa95c15b080 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa95c06b550 tx=0x7fa95402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8816554ac41d24d server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:03.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.796+0000 7fa95be00640 1 -- 172.21.15.143:0/447924631 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa954031860 con 0x7fa95c0cd470 2024-01-16T15:00:03.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.796+0000 7fa95be00640 1 -- 172.21.15.143:0/447924631 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9540319c0 con 0x7fa95c0cd470 2024-01-16T15:00:03.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.796+0000 7fa95be00640 1 -- 172.21.15.143:0/447924631 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa954031d50 con 0x7fa95c0cd470 2024-01-16T15:00:03.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.797+0000 7fa963e00640 1 -- 172.21.15.143:0/447924631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa95c0cd470 msgr2=0x7fa95c15b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:03.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.797+0000 7fa963e00640 1 --2- 172.21.15.143:0/447924631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa95c0cd470 0x7fa95c15b080 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa95c06b550 tx=0x7fa95402b900 comp rx=0 tx=0).stop 2024-01-16T15:00:03.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.797+0000 7fa963e00640 1 -- 172.21.15.143:0/447924631 shutdown_connections 2024-01-16T15:00:03.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.797+0000 7fa963e00640 1 --2- 172.21.15.143:0/447924631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa95c0cd470 0x7fa95c15b080 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:03.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.797+0000 7fa963e00640 1 --2- 172.21.15.143:0/447924631 >> v2:172.21.15.204:3300/0 conn(0x7fa95c0b2630 0x7fa95c0ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:03.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.797+0000 7fa963e00640 1 --2- 172.21.15.143:0/447924631 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa95c0b2250 0x7fa95c15df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:03.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.797+0000 7fa963e00640 1 -- 172.21.15.143:0/447924631 >> 172.21.15.143:0/447924631 conn(0x7fa95c06a920 msgr2=0x7fa95c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:03.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.797+0000 7fa963e00640 1 -- 172.21.15.143:0/447924631 shutdown_connections 2024-01-16T15:00:03.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.797+0000 7fa963e00640 1 -- 172.21.15.143:0/447924631 wait complete. 2024-01-16T15:00:03.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.799+0000 7fa963e00640 1 Processor -- start 2024-01-16T15:00:03.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.799+0000 7fa963e00640 1 -- start start 2024-01-16T15:00:03.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.800+0000 7fa963e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa95c0b2250 0x7fa95c145c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:03.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.800+0000 7fa963e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa95c0b2630 0x7fa95c1461d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:03.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.800+0000 7fa963e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa95c0cd470 0x7fa95c143dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:03.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.800+0000 7fa963e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c145770 con 0x7fa95c0b2630 2024-01-16T15:00:03.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.800+0000 7fa963e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c144b60 con 0x7fa95c0cd470 2024-01-16T15:00:03.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.800+0000 7fa963e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c144e40 con 0x7fa95c0b2250 2024-01-16T15:00:03.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.800+0000 7fa960a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa95c0b2630 0x7fa95c1461d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:03.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.800+0000 7fa960a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa95c0b2630 0x7fa95c1461d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57146/0 (socket says 172.21.15.143:57146) 2024-01-16T15:00:03.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.800+0000 7fa960a00640 1 -- 172.21.15.143:0/1271770822 learned_addr learned my addr 172.21.15.143:0/1271770822 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:03.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.800+0000 7fa961e00640 1 --2- 172.21.15.143:0/1271770822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa95c0cd470 0x7fa95c143dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:03.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.800+0000 7fa960a00640 1 -- 172.21.15.143:0/1271770822 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa95c0b2250 msgr2=0x7fa95c145c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:03.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.800+0000 7fa961400640 1 --2- 172.21.15.143:0/1271770822 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa95c0b2250 0x7fa95c145c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:03.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.800+0000 7fa960a00640 1 --2- 172.21.15.143:0/1271770822 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa95c0b2250 0x7fa95c145c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:03.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.800+0000 7fa960a00640 1 -- 172.21.15.143:0/1271770822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa95c0cd470 msgr2=0x7fa95c143dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:03.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.800+0000 7fa960a00640 1 --2- 172.21.15.143:0/1271770822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa95c0cd470 0x7fa95c143dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:03.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.800+0000 7fa960a00640 1 -- 172.21.15.143:0/1271770822 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9540094d0 con 0x7fa95c0b2630 2024-01-16T15:00:03.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.800+0000 7fa961e00640 1 --2- 172.21.15.143:0/1271770822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa95c0cd470 0x7fa95c143dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:03.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.800+0000 7fa961400640 1 --2- 172.21.15.143:0/1271770822 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa95c0b2250 0x7fa95c145c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:03.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.801+0000 7fa960a00640 1 --2- 172.21.15.143:0/1271770822 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa95c0b2630 0x7fa95c1461d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa94400d800 tx=0x7fa94400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:03.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.801+0000 7fa959600640 1 -- 172.21.15.143:0/1271770822 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94400b6c0 con 0x7fa95c0b2630 2024-01-16T15:00:03.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.801+0000 7fa959600640 1 -- 172.21.15.143:0/1271770822 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94400b820 con 0x7fa95c0b2630 2024-01-16T15:00:03.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.801+0000 7fa959600640 1 -- 172.21.15.143:0/1271770822 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94400bbb0 con 0x7fa95c0b2630 2024-01-16T15:00:03.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.801+0000 7fa963e00640 1 -- 172.21.15.143:0/1271770822 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa95c145120 con 0x7fa95c0b2630 2024-01-16T15:00:03.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.801+0000 7fa963e00640 1 -- 172.21.15.143:0/1271770822 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa95c149ff0 con 0x7fa95c0b2630 2024-01-16T15:00:03.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.802+0000 7fa963e00640 1 -- 172.21.15.143:0/1271770822 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa95c1458e0 con 0x7fa95c0b2630 2024-01-16T15:00:03.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.803+0000 7fa959600640 1 -- 172.21.15.143:0/1271770822 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fa94400bd10 con 0x7fa95c0b2630 2024-01-16T15:00:03.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.804+0000 7fa959600640 1 --2- 172.21.15.143:0/1271770822 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fa93406b2f0 0x7fa93406d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:03.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.804+0000 7fa959600640 1 -- 172.21.15.143:0/1271770822 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa94408a110 con 0x7fa95c0b2630 2024-01-16T15:00:03.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.804+0000 7fa961400640 1 --2- 172.21.15.143:0/1271770822 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fa93406b2f0 0x7fa93406d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:03.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.805+0000 7fa961400640 1 --2- 172.21.15.143:0/1271770822 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fa93406b2f0 0x7fa93406d7b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa95c0ce050 tx=0x7fa94c008530 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:03.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.809+0000 7fa959600640 1 -- 172.21.15.143:0/1271770822 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa944054f70 con 0x7fa95c0b2630 2024-01-16T15:00:03.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.983+0000 7fa963e00640 1 -- 172.21.15.143:0/1271770822 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa95c14a180 con 0x7fa95c0b2630 2024-01-16T15:00:03.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.985+0000 7fa959600640 1 -- 172.21.15.143:0/1271770822 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16578 (secure 0 0 0) 0x7fa944058590 con 0x7fa95c0b2630 2024-01-16T15:00:03.965 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:00:03.966 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":19,"fsid":"c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0","created":"2024-01-16T14:59:38.532601+0000","modified":"2024-01-16T15:00:03.049667+0000","last_up_change":"2024-01-16T14:59:50.983699+0000","last_in_change":"2024-01-16T14:59:42.056297+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:59:51.727720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T15:00:01.010472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T15:00:02.041228+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f4c8891d-ab29-472a-907d-8446f20f7012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6801","nonce":1569293626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6805","nonce":1569293626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6814","nonce":1569293626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6809","nonce":1569293626}]},"public_addr":"172.21.15.143:6801/1569293626","cluster_addr":"172.21.15.143:6805/1569293626","heartbeat_back_addr":"172.21.15.143:6814/1569293626","heartbeat_front_addr":"172.21.15.143:6809/1569293626","state":["exists","up"]},{"osd":1,"uuid":"c4a2788d-8b89-4992-82b2-7f4b9debd509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6804","nonce":1276841168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6808","nonce":1276841168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6813","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6815","nonce":1276841168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6811","nonce":1276841168}]},"public_addr":"172.21.15.143:6804/1276841168","cluster_addr":"172.21.15.143:6808/1276841168","heartbeat_back_addr":"172.21.15.143:6815/1276841168","heartbeat_front_addr":"172.21.15.143:6811/1276841168","state":["exists","up"]},{"osd":2,"uuid":"64645c1d-31b2-4794-b0f2-7ccf8df30393","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6817","nonce":1217670170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6819","nonce":1217670170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6823","nonce":1217670170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6821","nonce":1217670170}]},"public_addr":"172.21.15.143:6817/1217670170","cluster_addr":"172.21.15.143:6819/1217670170","heartbeat_back_addr":"172.21.15.143:6823/1217670170","heartbeat_front_addr":"172.21.15.143:6821/1217670170","state":["exists","up"]},{"osd":3,"uuid":"73674129-a84e-4d31-8247-7d0179ae6df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6825","nonce":3414188012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6827","nonce":3414188012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6831","nonce":3414188012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6829","nonce":3414188012}]},"public_addr":"172.21.15.143:6825/3414188012","cluster_addr":"172.21.15.143:6827/3414188012","heartbeat_back_addr":"172.21.15.143:6831/3414188012","heartbeat_front_addr":"172.21.15.143:6829/3414188012","state":["exists","up"]},{"osd":4,"uuid":"cea28c9b-2f1c-4087-b415-9960789131d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6811","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6812","nonce":1070255716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6816","nonce":1070255716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6823","nonce":1070255716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6820","nonce":1070255716}]},"public_addr":"172.21.15.204:6812/1070255716","cluster_addr":"172.21.15.204:6816/1070255716","heartbeat_back_addr":"172.21.15.204:6823/1070255716","heartbeat_front_addr":"172.21.15.204:6820/1070255716","state":["exists","up"]},{"osd":5,"uuid":"33b63560-5574-4e78-983f-b398d67a2cb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6809","nonce":3605417837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6813","nonce":3605417837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6819","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6821","nonce":3605417837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6817","nonce":3605417837}]},"public_addr":"172.21.15.204:6809/3605417837","cluster_addr":"172.21.15.204:6813/3605417837","heartbeat_back_addr":"172.21.15.204:6821/3605417837","heartbeat_front_addr":"172.21.15.204:6817/3605417837","state":["exists","up"]},{"osd":6,"uuid":"df397427-8940-4eee-9c79-eece669d735d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6801","nonce":1085000534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6803","nonce":1085000534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6808","nonce":1085000534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6805","nonce":1085000534}]},"public_addr":"172.21.15.204:6801/1085000534","cluster_addr":"172.21.15.204:6803/1085000534","heartbeat_back_addr":"172.21.15.204:6808/1085000534","heartbeat_front_addr":"172.21.15.204:6805/1085000534","state":["exists","up"]},{"osd":7,"uuid":"a2e5a15c-8312-405a-ac02-f7e1fc9df2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6825","nonce":1337220609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6827","nonce":1337220609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6831","nonce":1337220609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6829","nonce":1337220609}]},"public_addr":"172.21.15.204:6825/1337220609","cluster_addr":"172.21.15.204:6827/1337220609","heartbeat_back_addr":"172.21.15.204:6831/1337220609","heartbeat_front_addr":"172.21.15.204:6829/1337220609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:49.984507+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.395306+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.632998+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.373627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.342920+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.438366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.331926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.406657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T15:00:03.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.987+0000 7fa963e00640 1 -- 172.21.15.143:0/1271770822 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fa93406b2f0 msgr2=0x7fa93406d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:03.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.987+0000 7fa963e00640 1 --2- 172.21.15.143:0/1271770822 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fa93406b2f0 0x7fa93406d7b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa95c0ce050 tx=0x7fa94c008530 comp rx=0 tx=0).stop 2024-01-16T15:00:03.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.987+0000 7fa963e00640 1 -- 172.21.15.143:0/1271770822 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa95c0b2630 msgr2=0x7fa95c1461d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:03.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.987+0000 7fa963e00640 1 --2- 172.21.15.143:0/1271770822 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa95c0b2630 0x7fa95c1461d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa94400d800 tx=0x7fa94400dcc0 comp rx=0 tx=0).stop 2024-01-16T15:00:03.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.987+0000 7fa963e00640 1 -- 172.21.15.143:0/1271770822 shutdown_connections 2024-01-16T15:00:03.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.987+0000 7fa963e00640 1 --2- 172.21.15.143:0/1271770822 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fa93406b2f0 0x7fa93406d7b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:03.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.987+0000 7fa963e00640 1 --2- 172.21.15.143:0/1271770822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa95c0cd470 0x7fa95c143dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:03.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.987+0000 7fa963e00640 1 --2- 172.21.15.143:0/1271770822 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa95c0b2630 0x7fa95c1461d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:03.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.987+0000 7fa963e00640 1 --2- 172.21.15.143:0/1271770822 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa95c0b2250 0x7fa95c145c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:03.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.987+0000 7fa963e00640 1 -- 172.21.15.143:0/1271770822 >> 172.21.15.143:0/1271770822 conn(0x7fa95c06a920 msgr2=0x7fa95c15cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:03.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.988+0000 7fa963e00640 1 -- 172.21.15.143:0/1271770822 shutdown_connections 2024-01-16T15:00:03.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:03.988+0000 7fa963e00640 1 -- 172.21.15.143:0/1271770822 wait complete. 2024-01-16T15:00:03.984 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T15:00:04.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.194+0000 7f063b200640 1 Processor -- start 2024-01-16T15:00:04.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.195+0000 7f063b200640 1 -- start start 2024-01-16T15:00:04.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.196+0000 7f063b200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f063415af40 0x7f063406d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:04.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.196+0000 7f063b200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f063415b320 0x7f06340ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:04.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.196+0000 7f063b200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f063415e8b0 con 0x7f0634070c40 2024-01-16T15:00:04.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.196+0000 7f063b200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f063415eb90 con 0x7f063415b320 2024-01-16T15:00:04.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.196+0000 7f063b200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f063415e6a0 con 0x7f063415af40 2024-01-16T15:00:04.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.196+0000 7f062be00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f063415b320 0x7f06340ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:04.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.196+0000 7f062be00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f063415b320 0x7f06340ceec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57148/0 (socket says 172.21.15.143:57148) 2024-01-16T15:00:04.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.196+0000 7f062be00640 1 -- 172.21.15.143:0/806160336 learned_addr learned my addr 172.21.15.143:0/806160336 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:04.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.197+0000 7f0633e00640 1 --2- 172.21.15.143:0/806160336 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f063415af40 0x7f063406d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:04.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.197+0000 7f062be00640 1 -- 172.21.15.143:0/806160336 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f063415af40 msgr2=0x7f063406d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:04.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.197+0000 7f062be00640 1 --2- 172.21.15.143:0/806160336 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f063415af40 0x7f063406d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:04.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.197+0000 7f062be00640 1 -- 172.21.15.143:0/806160336 >> v1:172.21.15.204:6789/0 conn(0x7f0634070c40 legacy=0x7f0634161210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:04.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.197+0000 7f062be00640 1 -- 172.21.15.143:0/806160336 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0634070180 con 0x7f063415b320 2024-01-16T15:00:04.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.198+0000 7f062be00640 1 --2- 172.21.15.143:0/806160336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f063415b320 0x7f06340ceec0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f061c009870 tx=0x7f061c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b1950166953c2c server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:04.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.198+0000 7f0633400640 1 -- 172.21.15.143:0/806160336 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c030840 con 0x7f063415b320 2024-01-16T15:00:04.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.198+0000 7f0633400640 1 -- 172.21.15.143:0/806160336 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f061c0309a0 con 0x7f063415b320 2024-01-16T15:00:04.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.198+0000 7f0633400640 1 -- 172.21.15.143:0/806160336 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c030d30 con 0x7f063415b320 2024-01-16T15:00:04.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.199+0000 7f063b200640 1 -- 172.21.15.143:0/806160336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f063415b320 msgr2=0x7f06340ceec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:04.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.199+0000 7f063b200640 1 --2- 172.21.15.143:0/806160336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f063415b320 0x7f06340ceec0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f061c009870 tx=0x7f061c02b980 comp rx=0 tx=0).stop 2024-01-16T15:00:04.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.199+0000 7f063b200640 1 -- 172.21.15.143:0/806160336 shutdown_connections 2024-01-16T15:00:04.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.199+0000 7f063b200640 1 --2- 172.21.15.143:0/806160336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f063415b320 0x7f06340ceec0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:04.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.199+0000 7f063b200640 1 --2- 172.21.15.143:0/806160336 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f063415af40 0x7f063406d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:04.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.199+0000 7f063b200640 1 -- 172.21.15.143:0/806160336 >> 172.21.15.143:0/806160336 conn(0x7f063406e7f0 msgr2=0x7f063406ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:04.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.199+0000 7f063b200640 1 -- 172.21.15.143:0/806160336 shutdown_connections 2024-01-16T15:00:04.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.199+0000 7f063b200640 1 -- 172.21.15.143:0/806160336 wait complete. 2024-01-16T15:00:04.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.201+0000 7f063b200640 1 Processor -- start 2024-01-16T15:00:04.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.201+0000 7f063b200640 1 -- start start 2024-01-16T15:00:04.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.202+0000 7f063b200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0634070c40 0x7f06340b5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:04.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.202+0000 7f063b200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f063415af40 0x7f06340afb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:04.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.202+0000 7f063b200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f063415b320 0x7f06340b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:04.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.202+0000 7f063b200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06340ad3b0 con 0x7f063415b320 2024-01-16T15:00:04.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.202+0000 7f063b200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06340b05e0 con 0x7f063415af40 2024-01-16T15:00:04.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.202+0000 7f063b200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06340aca10 con 0x7f0634070c40 2024-01-16T15:00:04.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.202+0000 7f0639200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f063415b320 0x7f06340b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:04.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.202+0000 7f0639200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f063415b320 0x7f06340b00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57156/0 (socket says 172.21.15.143:57156) 2024-01-16T15:00:04.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.202+0000 7f0639200640 1 -- 172.21.15.143:0/1960492810 learned_addr learned my addr 172.21.15.143:0/1960492810 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:04.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.202+0000 7f062be00640 1 --2- 172.21.15.143:0/1960492810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f063415af40 0x7f06340afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:04.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.202+0000 7f0633e00640 1 --2- 172.21.15.143:0/1960492810 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0634070c40 0x7f06340b5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:04.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.203+0000 7f0639200640 1 -- 172.21.15.143:0/1960492810 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0634070c40 msgr2=0x7f06340b5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:04.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.203+0000 7f0639200640 1 --2- 172.21.15.143:0/1960492810 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0634070c40 0x7f06340b5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:04.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.203+0000 7f0639200640 1 -- 172.21.15.143:0/1960492810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f063415af40 msgr2=0x7f06340afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:04.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.203+0000 7f0639200640 1 --2- 172.21.15.143:0/1960492810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f063415af40 0x7f06340afb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:04.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.203+0000 7f0639200640 1 -- 172.21.15.143:0/1960492810 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f061c0094d0 con 0x7f063415b320 2024-01-16T15:00:04.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.203+0000 7f0633e00640 1 --2- 172.21.15.143:0/1960492810 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0634070c40 0x7f06340b5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:04.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.203+0000 7f0639200640 1 --2- 172.21.15.143:0/1960492810 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f063415b320 0x7f06340b00a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f062c00c6b0 tx=0x7f062c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:04.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.203+0000 7f0630c00640 1 -- 172.21.15.143:0/1960492810 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c003b70 con 0x7f063415b320 2024-01-16T15:00:04.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.203+0000 7f0630c00640 1 -- 172.21.15.143:0/1960492810 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062c003cf0 con 0x7f063415b320 2024-01-16T15:00:04.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.203+0000 7f0630c00640 1 -- 172.21.15.143:0/1960492810 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c003ea0 con 0x7f063415b320 2024-01-16T15:00:04.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.203+0000 7f063b200640 1 -- 172.21.15.143:0/1960492810 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06340accf0 con 0x7f063415b320 2024-01-16T15:00:04.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.203+0000 7f063b200640 1 -- 172.21.15.143:0/1960492810 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0634159ef0 con 0x7f063415b320 2024-01-16T15:00:04.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.204+0000 7f063b200640 1 -- 172.21.15.143:0/1960492810 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f063415a080 con 0x7f063415b320 2024-01-16T15:00:04.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.205+0000 7f0630c00640 1 -- 172.21.15.143:0/1960492810 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f062c005310 con 0x7f063415b320 2024-01-16T15:00:04.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.206+0000 7f0630c00640 1 --2- 172.21.15.143:0/1960492810 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f060c06b2c0 0x7f060c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:04.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.206+0000 7f0630c00640 1 -- 172.21.15.143:0/1960492810 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f062c0898f0 con 0x7f063415b320 2024-01-16T15:00:04.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.206+0000 7f0633e00640 1 --2- 172.21.15.143:0/1960492810 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f060c06b2c0 0x7f060c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:04.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.207+0000 7f0633e00640 1 --2- 172.21.15.143:0/1960492810 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f060c06b2c0 0x7f060c06d780 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0624006fd0 tx=0x7f0624008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:04.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.210+0000 7f0630c00640 1 -- 172.21.15.143:0/1960492810 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f062c0547f0 con 0x7f063415b320 2024-01-16T15:00:04.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.400+0000 7f063b200640 1 -- 172.21.15.143:0/1960492810 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06340ad4f0 con 0x7f063415b320 2024-01-16T15:00:04.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.402+0000 7f0630c00640 1 -- 172.21.15.143:0/1960492810 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16578 (secure 0 0 0) 0x7f062c057e10 con 0x7f063415b320 2024-01-16T15:00:04.382 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:00:04.383 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":19,"fsid":"c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0","created":"2024-01-16T14:59:38.532601+0000","modified":"2024-01-16T15:00:03.049667+0000","last_up_change":"2024-01-16T14:59:50.983699+0000","last_in_change":"2024-01-16T14:59:42.056297+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:59:51.727720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T15:00:01.010472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T15:00:02.041228+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f4c8891d-ab29-472a-907d-8446f20f7012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6801","nonce":1569293626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6805","nonce":1569293626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6814","nonce":1569293626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6809","nonce":1569293626}]},"public_addr":"172.21.15.143:6801/1569293626","cluster_addr":"172.21.15.143:6805/1569293626","heartbeat_back_addr":"172.21.15.143:6814/1569293626","heartbeat_front_addr":"172.21.15.143:6809/1569293626","state":["exists","up"]},{"osd":1,"uuid":"c4a2788d-8b89-4992-82b2-7f4b9debd509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6804","nonce":1276841168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6808","nonce":1276841168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6813","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6815","nonce":1276841168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6811","nonce":1276841168}]},"public_addr":"172.21.15.143:6804/1276841168","cluster_addr":"172.21.15.143:6808/1276841168","heartbeat_back_addr":"172.21.15.143:6815/1276841168","heartbeat_front_addr":"172.21.15.143:6811/1276841168","state":["exists","up"]},{"osd":2,"uuid":"64645c1d-31b2-4794-b0f2-7ccf8df30393","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6817","nonce":1217670170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6819","nonce":1217670170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6823","nonce":1217670170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6821","nonce":1217670170}]},"public_addr":"172.21.15.143:6817/1217670170","cluster_addr":"172.21.15.143:6819/1217670170","heartbeat_back_addr":"172.21.15.143:6823/1217670170","heartbeat_front_addr":"172.21.15.143:6821/1217670170","state":["exists","up"]},{"osd":3,"uuid":"73674129-a84e-4d31-8247-7d0179ae6df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6825","nonce":3414188012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6827","nonce":3414188012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6831","nonce":3414188012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6829","nonce":3414188012}]},"public_addr":"172.21.15.143:6825/3414188012","cluster_addr":"172.21.15.143:6827/3414188012","heartbeat_back_addr":"172.21.15.143:6831/3414188012","heartbeat_front_addr":"172.21.15.143:6829/3414188012","state":["exists","up"]},{"osd":4,"uuid":"cea28c9b-2f1c-4087-b415-9960789131d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6811","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6812","nonce":1070255716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6816","nonce":1070255716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6823","nonce":1070255716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6820","nonce":1070255716}]},"public_addr":"172.21.15.204:6812/1070255716","cluster_addr":"172.21.15.204:6816/1070255716","heartbeat_back_addr":"172.21.15.204:6823/1070255716","heartbeat_front_addr":"172.21.15.204:6820/1070255716","state":["exists","up"]},{"osd":5,"uuid":"33b63560-5574-4e78-983f-b398d67a2cb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6809","nonce":3605417837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6813","nonce":3605417837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6819","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6821","nonce":3605417837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6817","nonce":3605417837}]},"public_addr":"172.21.15.204:6809/3605417837","cluster_addr":"172.21.15.204:6813/3605417837","heartbeat_back_addr":"172.21.15.204:6821/3605417837","heartbeat_front_addr":"172.21.15.204:6817/3605417837","state":["exists","up"]},{"osd":6,"uuid":"df397427-8940-4eee-9c79-eece669d735d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6801","nonce":1085000534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6803","nonce":1085000534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6808","nonce":1085000534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6805","nonce":1085000534}]},"public_addr":"172.21.15.204:6801/1085000534","cluster_addr":"172.21.15.204:6803/1085000534","heartbeat_back_addr":"172.21.15.204:6808/1085000534","heartbeat_front_addr":"172.21.15.204:6805/1085000534","state":["exists","up"]},{"osd":7,"uuid":"a2e5a15c-8312-405a-ac02-f7e1fc9df2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6825","nonce":1337220609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6827","nonce":1337220609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6831","nonce":1337220609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6829","nonce":1337220609}]},"public_addr":"172.21.15.204:6825/1337220609","cluster_addr":"172.21.15.204:6827/1337220609","heartbeat_back_addr":"172.21.15.204:6831/1337220609","heartbeat_front_addr":"172.21.15.204:6829/1337220609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:49.984507+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.395306+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.632998+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.373627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.342920+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.438366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.331926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.406657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T15:00:04.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.405+0000 7f063b200640 1 -- 172.21.15.143:0/1960492810 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f060c06b2c0 msgr2=0x7f060c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:04.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.405+0000 7f063b200640 1 --2- 172.21.15.143:0/1960492810 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f060c06b2c0 0x7f060c06d780 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0624006fd0 tx=0x7f0624008040 comp rx=0 tx=0).stop 2024-01-16T15:00:04.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.405+0000 7f063b200640 1 -- 172.21.15.143:0/1960492810 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f063415b320 msgr2=0x7f06340b00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:04.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.405+0000 7f063b200640 1 --2- 172.21.15.143:0/1960492810 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f063415b320 0x7f06340b00a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f062c00c6b0 tx=0x7f062c00cb70 comp rx=0 tx=0).stop 2024-01-16T15:00:04.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.405+0000 7f063b200640 1 -- 172.21.15.143:0/1960492810 shutdown_connections 2024-01-16T15:00:04.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.405+0000 7f063b200640 1 --2- 172.21.15.143:0/1960492810 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f060c06b2c0 0x7f060c06d780 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:04.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.405+0000 7f063b200640 1 --2- 172.21.15.143:0/1960492810 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f063415b320 0x7f06340b00a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:04.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.405+0000 7f063b200640 1 --2- 172.21.15.143:0/1960492810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f063415af40 0x7f06340afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:04.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.405+0000 7f063b200640 1 --2- 172.21.15.143:0/1960492810 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0634070c40 0x7f06340b5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:04.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.405+0000 7f063b200640 1 -- 172.21.15.143:0/1960492810 >> 172.21.15.143:0/1960492810 conn(0x7f063406e7f0 msgr2=0x7f06340ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:04.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.405+0000 7f063b200640 1 -- 172.21.15.143:0/1960492810 shutdown_connections 2024-01-16T15:00:04.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.405+0000 7f063b200640 1 -- 172.21.15.143:0/1960492810 wait complete. 2024-01-16T15:00:04.404 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-16T15:00:04.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.633+0000 7f3faf200640 1 Processor -- start 2024-01-16T15:00:04.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.633+0000 7f3faf200640 1 -- start start 2024-01-16T15:00:04.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.634+0000 7f3faf200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3fa8146eb0 0x7f3fa8161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:04.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.635+0000 7f3faf200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3fa806e160 0x7f3fa80ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:04.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.635+0000 7f3faf200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fa80cd530 con 0x7f3fa80cd110 2024-01-16T15:00:04.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.635+0000 7f3faf200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fa8161a10 con 0x7f3fa806e160 2024-01-16T15:00:04.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.635+0000 7f3faf200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa8161cf0 con 0x7f3fa8146eb0 2024-01-16T15:00:04.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.635+0000 7f3fa7e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3fa8146eb0 0x7f3fa8161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:04.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.635+0000 7f3fa7e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3fa8146eb0 0x7f3fa8161150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:43260/0 (socket says 172.21.15.143:43260) 2024-01-16T15:00:04.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.635+0000 7f3fa7e00640 1 -- 172.21.15.143:0/717178010 learned_addr learned my addr 172.21.15.143:0/717178010 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:04.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.635+0000 7f3fa7400640 1 --2- 172.21.15.143:0/717178010 >> v2:172.21.15.204:3300/0 conn(0x7f3fa806e160 0x7f3fa80ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:04.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.636+0000 7f3fa7e00640 1 -- 172.21.15.143:0/717178010 >> v2:172.21.15.204:3300/0 conn(0x7f3fa806e160 msgr2=0x7f3fa80ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:04.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.636+0000 7f3fa7e00640 1 --2- 172.21.15.143:0/717178010 >> v2:172.21.15.204:3300/0 conn(0x7f3fa806e160 0x7f3fa80ccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:04.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.636+0000 7f3fa7e00640 1 -- 172.21.15.143:0/717178010 >> v1:172.21.15.204:6789/0 conn(0x7f3fa80cd110 legacy=0x7f3fa806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:04.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.636+0000 7f3fa7e00640 1 -- 172.21.15.143:0/717178010 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa8161fd0 con 0x7f3fa8146eb0 2024-01-16T15:00:04.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.638+0000 7f3fa7e00640 1 --2- 172.21.15.143:0/717178010 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3fa8146eb0 0x7f3fa8161150 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3f98009870 tx=0x7f3f9802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7a527f17931e00e server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:04.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.638+0000 7f3fa6a00640 1 -- 172.21.15.143:0/717178010 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f98030840 con 0x7f3fa8146eb0 2024-01-16T15:00:04.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.638+0000 7f3fa6a00640 1 -- 172.21.15.143:0/717178010 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f980309a0 con 0x7f3fa8146eb0 2024-01-16T15:00:04.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.638+0000 7f3fa6a00640 1 -- 172.21.15.143:0/717178010 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f98030d30 con 0x7f3fa8146eb0 2024-01-16T15:00:04.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.639+0000 7f3faf200640 1 -- 172.21.15.143:0/717178010 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3fa8146eb0 msgr2=0x7f3fa8161150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:04.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.639+0000 7f3faf200640 1 --2- 172.21.15.143:0/717178010 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3fa8146eb0 0x7f3fa8161150 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3f98009870 tx=0x7f3f9802b980 comp rx=0 tx=0).stop 2024-01-16T15:00:04.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.639+0000 7f3faf200640 1 -- 172.21.15.143:0/717178010 shutdown_connections 2024-01-16T15:00:04.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.639+0000 7f3faf200640 1 --2- 172.21.15.143:0/717178010 >> v2:172.21.15.204:3300/0 conn(0x7f3fa806e160 0x7f3fa80ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:04.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.639+0000 7f3faf200640 1 --2- 172.21.15.143:0/717178010 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3fa8146eb0 0x7f3fa8161150 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:04.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.639+0000 7f3faf200640 1 -- 172.21.15.143:0/717178010 >> 172.21.15.143:0/717178010 conn(0x7f3fa80b0c50 msgr2=0x7f3fa80b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:04.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.639+0000 7f3faf200640 1 -- 172.21.15.143:0/717178010 shutdown_connections 2024-01-16T15:00:04.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.639+0000 7f3faf200640 1 -- 172.21.15.143:0/717178010 wait complete. 2024-01-16T15:00:04.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.641+0000 7f3faf200640 1 Processor -- start 2024-01-16T15:00:04.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.641+0000 7f3faf200640 1 -- start start 2024-01-16T15:00:04.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.642+0000 7f3faf200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3fa806e160 0x7f3fa8142c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:04.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.642+0000 7f3faf200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fa80cd110 0x7f3fa8145ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:04.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.642+0000 7f3faf200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fa8146eb0 0x7f3fa8146000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:04.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.643+0000 7f3faf200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa81f3da0 con 0x7f3fa8146eb0 2024-01-16T15:00:04.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.643+0000 7f3faf200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa81f3120 con 0x7f3fa80cd110 2024-01-16T15:00:04.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.643+0000 7f3faf200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa81f3400 con 0x7f3fa806e160 2024-01-16T15:00:04.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.643+0000 7f3fad200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fa8146eb0 0x7f3fa8146000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:04.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.643+0000 7f3fad200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fa8146eb0 0x7f3fa8146000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57162/0 (socket says 172.21.15.143:57162) 2024-01-16T15:00:04.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.643+0000 7f3fad200640 1 -- 172.21.15.143:0/3782125748 learned_addr learned my addr 172.21.15.143:0/3782125748 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:04.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.643+0000 7f3fa7e00640 1 --2- 172.21.15.143:0/3782125748 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3fa806e160 0x7f3fa8142c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:04.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.643+0000 7f3fad200640 1 -- 172.21.15.143:0/3782125748 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3fa806e160 msgr2=0x7f3fa8142c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:04.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.643+0000 7f3fad200640 1 --2- 172.21.15.143:0/3782125748 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3fa806e160 0x7f3fa8142c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:04.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.643+0000 7f3fad200640 1 -- 172.21.15.143:0/3782125748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fa80cd110 msgr2=0x7f3fa8145ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:04.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.643+0000 7f3fad200640 1 --2- 172.21.15.143:0/3782125748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fa80cd110 0x7f3fa8145ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:04.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.643+0000 7f3fad200640 1 -- 172.21.15.143:0/3782125748 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f980094d0 con 0x7f3fa8146eb0 2024-01-16T15:00:04.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.643+0000 7f3fad200640 1 --2- 172.21.15.143:0/3782125748 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fa8146eb0 0x7f3fa8146000 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3fa000c7a0 tx=0x7f3fa000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:04.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.644+0000 7f3f9fe00640 1 -- 172.21.15.143:0/3782125748 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0003b70 con 0x7f3fa8146eb0 2024-01-16T15:00:04.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.644+0000 7f3f9fe00640 1 -- 172.21.15.143:0/3782125748 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa0003cf0 con 0x7f3fa8146eb0 2024-01-16T15:00:04.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.644+0000 7f3f9fe00640 1 -- 172.21.15.143:0/3782125748 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0003ea0 con 0x7f3fa8146eb0 2024-01-16T15:00:04.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.644+0000 7f3faf200640 1 -- 172.21.15.143:0/3782125748 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fa81f36e0 con 0x7f3fa8146eb0 2024-01-16T15:00:04.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.644+0000 7f3faf200640 1 -- 172.21.15.143:0/3782125748 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fa8143d60 con 0x7f3fa8146eb0 2024-01-16T15:00:04.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.645+0000 7f3faf200640 1 -- 172.21.15.143:0/3782125748 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fa81f3ee0 con 0x7f3fa8146eb0 2024-01-16T15:00:04.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.646+0000 7f3f9fe00640 1 -- 172.21.15.143:0/3782125748 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f3fa000aa90 con 0x7f3fa8146eb0 2024-01-16T15:00:04.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.647+0000 7f3f9fe00640 1 --2- 172.21.15.143:0/3782125748 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f3f8006b2c0 0x7f3f8006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:04.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.647+0000 7f3f9fe00640 1 -- 172.21.15.143:0/3782125748 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3fa0089b80 con 0x7f3fa8146eb0 2024-01-16T15:00:04.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.647+0000 7f3fa7e00640 1 --2- 172.21.15.143:0/3782125748 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f3f8006b2c0 0x7f3f8006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:04.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.648+0000 7f3fa7e00640 1 --2- 172.21.15.143:0/3782125748 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f3f8006b2c0 0x7f3f8006d780 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3f98036000 tx=0x7f3f9803b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:04.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.652+0000 7f3f9fe00640 1 -- 172.21.15.143:0/3782125748 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3fa81f3ee0 con 0x7f3fa8146eb0 2024-01-16T15:00:04.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:04.879+0000 7f3faf200640 1 -- 172.21.15.143:0/3782125748 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f3fa81f3ee0 con 0x7f3fa8146eb0 2024-01-16T15:00:05.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.626+0000 7f3f9fe00640 1 -- 172.21.15.143:0/3782125748 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3fa81f3ee0 con 0x7f3fa8146eb0 2024-01-16T15:00:05.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.629+0000 7f3faf200640 1 -- 172.21.15.143:0/3782125748 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f3f8006b2c0 msgr2=0x7f3f8006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:05.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.629+0000 7f3faf200640 1 --2- 172.21.15.143:0/3782125748 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f3f8006b2c0 0x7f3f8006d780 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3f98036000 tx=0x7f3f9803b040 comp rx=0 tx=0).stop 2024-01-16T15:00:05.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.629+0000 7f3faf200640 1 -- 172.21.15.143:0/3782125748 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fa8146eb0 msgr2=0x7f3fa8146000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:05.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.629+0000 7f3faf200640 1 --2- 172.21.15.143:0/3782125748 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fa8146eb0 0x7f3fa8146000 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3fa000c7a0 tx=0x7f3fa000cc60 comp rx=0 tx=0).stop 2024-01-16T15:00:05.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.630+0000 7f3faf200640 1 -- 172.21.15.143:0/3782125748 shutdown_connections 2024-01-16T15:00:05.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.630+0000 7f3faf200640 1 --2- 172.21.15.143:0/3782125748 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f3f8006b2c0 0x7f3f8006d780 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:05.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.630+0000 7f3faf200640 1 --2- 172.21.15.143:0/3782125748 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fa8146eb0 0x7f3fa8146000 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:05.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.630+0000 7f3faf200640 1 --2- 172.21.15.143:0/3782125748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fa80cd110 0x7f3fa8145ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:05.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.630+0000 7f3faf200640 1 --2- 172.21.15.143:0/3782125748 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3fa806e160 0x7f3fa8142c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:05.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.630+0000 7f3faf200640 1 -- 172.21.15.143:0/3782125748 >> 172.21.15.143:0/3782125748 conn(0x7f3fa80b0c50 msgr2=0x7f3fa80b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:05.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.630+0000 7f3faf200640 1 -- 172.21.15.143:0/3782125748 shutdown_connections 2024-01-16T15:00:05.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.630+0000 7f3faf200640 1 -- 172.21.15.143:0/3782125748 wait complete. 2024-01-16T15:00:05.632 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-01-16T15:00:05.633 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-16T15:00:05.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.860+0000 7fbec0c00640 1 Processor -- start 2024-01-16T15:00:05.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.860+0000 7fbec0c00640 1 -- start start 2024-01-16T15:00:05.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.861+0000 7fbec0c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbebc0b0b40 0x7fbebc0b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:05.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.862+0000 7fbec0c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbebc15ed20 con 0x7fbebc06c5b0 2024-01-16T15:00:05.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.862+0000 7fbec0c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbebc15f8a0 con 0x7fbebc15b260 2024-01-16T15:00:05.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.862+0000 7fbec0c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbebc15b640 con 0x7fbebc0b0b40 2024-01-16T15:00:05.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.862+0000 7fbebbe00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fbebc06c5b0 0x7fbebc06ca20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54048/0 (socket says 172.21.15.143:54048) 2024-01-16T15:00:05.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.862+0000 7fbebbe00640 1 -- 172.21.15.143:0/2706949357 learned_addr learned my addr 172.21.15.143:0/2706949357 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:05.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.862+0000 7fbeb3e00640 1 --2- 172.21.15.143:0/2706949357 >> v2:172.21.15.204:3300/0 conn(0x7fbebc0b0b40 0x7fbebc0b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:05.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.862+0000 7fbebaa00640 1 -- 172.21.15.143:0/2706949357 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2705713125 0 0) 0x7fbebc15ed20 con 0x7fbebc06c5b0 2024-01-16T15:00:05.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.862+0000 7fbebaa00640 1 -- 172.21.15.143:0/2706949357 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe9c003520 con 0x7fbebc06c5b0 2024-01-16T15:00:05.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.863+0000 7fbebaa00640 1 -- 172.21.15.143:0/2706949357 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1378017442 0 0) 0x7fbe9c003520 con 0x7fbebc06c5b0 2024-01-16T15:00:05.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.863+0000 7fbebaa00640 1 -- 172.21.15.143:0/2706949357 >> v2:172.21.15.204:3300/0 conn(0x7fbebc0b0b40 msgr2=0x7fbebc0b0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:05.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.863+0000 7fbebaa00640 1 --2- 172.21.15.143:0/2706949357 >> v2:172.21.15.204:3300/0 conn(0x7fbebc0b0b40 0x7fbebc0b0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:05.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.863+0000 7fbebaa00640 1 -- 172.21.15.143:0/2706949357 >> v1:172.21.15.204:6789/0 conn(0x7fbebc15b260 legacy=0x7fbebc06e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:05.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.863+0000 7fbebaa00640 1 -- 172.21.15.143:0/2706949357 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbebc06e750 con 0x7fbebc06c5b0 2024-01-16T15:00:05.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.863+0000 7fbebaa00640 1 -- 172.21.15.143:0/2706949357 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7fbeb40019f0 con 0x7fbebc06c5b0 2024-01-16T15:00:05.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.863+0000 7fbebaa00640 1 -- 172.21.15.143:0/2706949357 >> v1:172.21.15.143:6789/0 conn(0x7fbebc06c5b0 legacy=0x7fbebc06ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:05.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.863+0000 7fbebaa00640 1 --2- 172.21.15.143:0/2706949357 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe9c0039c0 0x7fbe9c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:05.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.863+0000 7fbebaa00640 1 --2- 172.21.15.143:0/2706949357 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe9c0047c0 0x7fbe9c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:05.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.863+0000 7fbebaa00640 1 --2- 172.21.15.143:0/2706949357 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe9c0052a0 0x7fbe9c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:05.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.863+0000 7fbebaa00640 1 -- 172.21.15.143:0/2706949357 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc15b640 con 0x7fbe9c0047c0 2024-01-16T15:00:05.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.863+0000 7fbebaa00640 1 -- 172.21.15.143:0/2706949357 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c00c1a0 con 0x7fbe9c0052a0 2024-01-16T15:00:05.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.863+0000 7fbebaa00640 1 -- 172.21.15.143:0/2706949357 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c00c360 con 0x7fbe9c0039c0 2024-01-16T15:00:05.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.863+0000 7fbeb3e00640 1 --2- 172.21.15.143:0/2706949357 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe9c0047c0 0x7fbe9c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:05.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.864+0000 7fbebb400640 1 --2- 172.21.15.143:0/2706949357 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe9c0039c0 0x7fbe9c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:05.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.864+0000 7fbeb3e00640 1 -- 172.21.15.143:0/2706949357 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe9c0039c0 msgr2=0x7fbe9c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:05.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.864+0000 7fbeb3e00640 1 --2- 172.21.15.143:0/2706949357 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe9c0039c0 0x7fbe9c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:05.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.864+0000 7fbebbe00640 1 --2- 172.21.15.143:0/2706949357 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe9c0052a0 0x7fbe9c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:05.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.864+0000 7fbeb3e00640 1 -- 172.21.15.143:0/2706949357 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe9c0052a0 msgr2=0x7fbe9c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:05.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.864+0000 7fbeb3e00640 1 --2- 172.21.15.143:0/2706949357 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe9c0052a0 0x7fbe9c00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:05.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.864+0000 7fbebbe00640 1 -- 172.21.15.143:0/2706949357 reap_dead start 2024-01-16T15:00:05.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.864+0000 7fbeb3e00640 1 -- 172.21.15.143:0/2706949357 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe9c003520 con 0x7fbe9c0047c0 2024-01-16T15:00:05.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.864+0000 7fbebb400640 1 --2- 172.21.15.143:0/2706949357 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe9c0039c0 0x7fbe9c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:05.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.865+0000 7fbeb3e00640 1 --2- 172.21.15.143:0/2706949357 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe9c0047c0 0x7fbe9c004c40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbea4009ac0 tx=0x7fbe9c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=947abe02bab8d9b7 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:05.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.865+0000 7fbebaa00640 1 -- 172.21.15.143:0/2706949357 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea402c6b0 con 0x7fbe9c0047c0 2024-01-16T15:00:05.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.865+0000 7fbebaa00640 1 -- 172.21.15.143:0/2706949357 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea402c810 con 0x7fbe9c0047c0 2024-01-16T15:00:05.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.865+0000 7fbebaa00640 1 -- 172.21.15.143:0/2706949357 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea402cba0 con 0x7fbe9c0047c0 2024-01-16T15:00:05.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.866+0000 7fbec0c00640 1 -- 172.21.15.143:0/2706949357 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe9c0047c0 msgr2=0x7fbe9c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:05.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.866+0000 7fbec0c00640 1 --2- 172.21.15.143:0/2706949357 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe9c0047c0 0x7fbe9c004c40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbea4009ac0 tx=0x7fbe9c0057f0 comp rx=0 tx=0).stop 2024-01-16T15:00:05.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.866+0000 7fbec0c00640 1 -- 172.21.15.143:0/2706949357 shutdown_connections 2024-01-16T15:00:05.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.866+0000 7fbec0c00640 1 --2- 172.21.15.143:0/2706949357 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe9c0047c0 0x7fbe9c004c40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:05.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.866+0000 7fbec0c00640 1 -- 172.21.15.143:0/2706949357 >> 172.21.15.143:0/2706949357 conn(0x7fbebc0cbb40 msgr2=0x7fbebc0cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:05.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.866+0000 7fbec0c00640 1 -- 172.21.15.143:0/2706949357 shutdown_connections 2024-01-16T15:00:05.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.866+0000 7fbec0c00640 1 -- 172.21.15.143:0/2706949357 wait complete. 2024-01-16T15:00:05.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.868+0000 7fbec0c00640 1 Processor -- start 2024-01-16T15:00:05.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.868+0000 7fbec0c00640 1 -- start start 2024-01-16T15:00:05.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.869+0000 7fbec0c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe9c0047c0 0x7fbebc154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:05.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.869+0000 7fbec0c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbebc1553d0 0x7fbebc0b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:05.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.869+0000 7fbec0c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbebc0ad020 0x7fbebc0ad4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:05.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.869+0000 7fbec0c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc099a80 con 0x7fbebc1553d0 2024-01-16T15:00:05.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.869+0000 7fbec0c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc155230 con 0x7fbebc0ad020 2024-01-16T15:00:05.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.869+0000 7fbec0c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc0c2bd0 con 0x7fbe9c0047c0 2024-01-16T15:00:05.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.869+0000 7fbeb3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbebc1553d0 0x7fbebc0b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:05.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.869+0000 7fbeb3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbebc1553d0 0x7fbebc0b3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42358/0 (socket says 172.21.15.143:42358) 2024-01-16T15:00:05.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.869+0000 7fbeb3e00640 1 -- 172.21.15.143:0/1037553535 learned_addr learned my addr 172.21.15.143:0/1037553535 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:05.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.869+0000 7fbebb400640 1 --2- 172.21.15.143:0/1037553535 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe9c0047c0 0x7fbebc154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:05.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.869+0000 7fbebbe00640 1 --2- 172.21.15.143:0/1037553535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbebc0ad020 0x7fbebc0ad4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:05.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.870+0000 7fbeb3e00640 1 -- 172.21.15.143:0/1037553535 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe9c0047c0 msgr2=0x7fbebc154cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:05.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.870+0000 7fbeb3e00640 1 --2- 172.21.15.143:0/1037553535 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe9c0047c0 0x7fbebc154cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:05.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.870+0000 7fbeb3e00640 1 -- 172.21.15.143:0/1037553535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbebc0ad020 msgr2=0x7fbebc0ad4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:05.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.870+0000 7fbeb3e00640 1 --2- 172.21.15.143:0/1037553535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbebc0ad020 0x7fbebc0ad4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:05.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.870+0000 7fbeb3e00640 1 -- 172.21.15.143:0/1037553535 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbebc0c3190 con 0x7fbebc1553d0 2024-01-16T15:00:05.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.870+0000 7fbebbe00640 1 --2- 172.21.15.143:0/1037553535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbebc0ad020 0x7fbebc0ad4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:05.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.870+0000 7fbeb3e00640 1 --2- 172.21.15.143:0/1037553535 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbebc1553d0 0x7fbebc0b3f60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbea4037040 tx=0x7fbea4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:05.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.870+0000 7fbeb3400640 1 -- 172.21.15.143:0/1037553535 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea4003810 con 0x7fbebc1553d0 2024-01-16T15:00:05.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.870+0000 7fbeb3400640 1 -- 172.21.15.143:0/1037553535 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea4003970 con 0x7fbebc1553d0 2024-01-16T15:00:05.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.870+0000 7fbeb3400640 1 -- 172.21.15.143:0/1037553535 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea4003d20 con 0x7fbebc1553d0 2024-01-16T15:00:05.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.870+0000 7fbec0c00640 1 -- 172.21.15.143:0/1037553535 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbebc0ad9e0 con 0x7fbebc1553d0 2024-01-16T15:00:05.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.870+0000 7fbec0c00640 1 -- 172.21.15.143:0/1037553535 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbebc0b4720 con 0x7fbebc1553d0 2024-01-16T15:00:05.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.871+0000 7fbec0c00640 1 -- 172.21.15.143:0/1037553535 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbebc04e840 con 0x7fbebc1553d0 2024-01-16T15:00:05.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.872+0000 7fbeb3400640 1 -- 172.21.15.143:0/1037553535 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fbea4034440 con 0x7fbebc1553d0 2024-01-16T15:00:05.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.873+0000 7fbeb3400640 1 --2- 172.21.15.143:0/1037553535 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbe9406b2c0 0x7fbe9406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:05.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.873+0000 7fbeb3400640 1 -- 172.21.15.143:0/1037553535 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbea40abcc0 con 0x7fbebc1553d0 2024-01-16T15:00:05.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.873+0000 7fbebb400640 1 --2- 172.21.15.143:0/1037553535 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbe9406b2c0 0x7fbe9406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:05.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.874+0000 7fbebb400640 1 --2- 172.21.15.143:0/1037553535 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbe9406b2c0 0x7fbe9406d780 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbebc064320 tx=0x7fbeac009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:05.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:05.878+0000 7fbeb3400640 1 -- 172.21.15.143:0/1037553535 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbea40b2080 con 0x7fbebc1553d0 2024-01-16T15:00:06.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.108+0000 7fbec0c00640 1 -- 172.21.15.143:0/1037553535 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fbebc0cc250 con 0x7fbebc1553d0 2024-01-16T15:00:06.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.629+0000 7fbeb3400640 1 -- 172.21.15.143:0/1037553535 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbea40390c0 con 0x7fbebc1553d0 2024-01-16T15:00:06.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.633+0000 7fbec0c00640 1 -- 172.21.15.143:0/1037553535 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbe9406b2c0 msgr2=0x7fbe9406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:06.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.633+0000 7fbec0c00640 1 --2- 172.21.15.143:0/1037553535 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbe9406b2c0 0x7fbe9406d780 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbebc064320 tx=0x7fbeac009040 comp rx=0 tx=0).stop 2024-01-16T15:00:06.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.633+0000 7fbec0c00640 1 -- 172.21.15.143:0/1037553535 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbebc1553d0 msgr2=0x7fbebc0b3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:06.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.633+0000 7fbec0c00640 1 --2- 172.21.15.143:0/1037553535 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbebc1553d0 0x7fbebc0b3f60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbea4037040 tx=0x7fbea4004730 comp rx=0 tx=0).stop 2024-01-16T15:00:06.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.634+0000 7fbec0c00640 1 -- 172.21.15.143:0/1037553535 shutdown_connections 2024-01-16T15:00:06.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.634+0000 7fbec0c00640 1 --2- 172.21.15.143:0/1037553535 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbe9406b2c0 0x7fbe9406d780 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:06.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.634+0000 7fbec0c00640 1 --2- 172.21.15.143:0/1037553535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbebc0ad020 0x7fbebc0ad4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:06.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.634+0000 7fbec0c00640 1 --2- 172.21.15.143:0/1037553535 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbebc1553d0 0x7fbebc0b3f60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:06.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.634+0000 7fbec0c00640 1 --2- 172.21.15.143:0/1037553535 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe9c0047c0 0x7fbebc154cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:06.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.634+0000 7fbec0c00640 1 -- 172.21.15.143:0/1037553535 >> 172.21.15.143:0/1037553535 conn(0x7fbebc0cbb40 msgr2=0x7fbebc0af530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:06.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.634+0000 7fbec0c00640 1 -- 172.21.15.143:0/1037553535 shutdown_connections 2024-01-16T15:00:06.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.634+0000 7fbec0c00640 1 -- 172.21.15.143:0/1037553535 wait complete. 2024-01-16T15:00:06.636 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-16T15:00:06.637 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-16T15:00:06.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.861+0000 7f0cf5800640 1 Processor -- start 2024-01-16T15:00:06.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.861+0000 7f0cf5800640 1 -- start start 2024-01-16T15:00:06.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.863+0000 7f0cf5800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0cf015c410 0x7f0cf015f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:06.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.863+0000 7f0cf5800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0cf00b1de0 0x7f0cf00b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:06.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.863+0000 7f0cf5800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf015bf80 con 0x7f0cf0144b30 2024-01-16T15:00:06.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.863+0000 7f0cf5800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cf00b27b0 con 0x7f0cf00b1de0 2024-01-16T15:00:06.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.863+0000 7f0cf5800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cf00b1560 con 0x7f0cf015c410 2024-01-16T15:00:06.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.863+0000 7f0cefe00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f0cf0144b30 0x7f0cf00b0a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54054/0 (socket says 172.21.15.143:54054) 2024-01-16T15:00:06.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.863+0000 7f0cefe00640 1 -- 172.21.15.143:0/2977788220 learned_addr learned my addr 172.21.15.143:0/2977788220 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:06.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.863+0000 7f0cf4e00640 1 --2- 172.21.15.143:0/2977788220 >> v2:172.21.15.143:3300/0 conn(0x7f0cf00b1de0 0x7f0cf00b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:06.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.863+0000 7f0ce7e00640 1 --2- 172.21.15.143:0/2977788220 >> v2:172.21.15.204:3300/0 conn(0x7f0cf015c410 0x7f0cf015f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:06.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.863+0000 7f0cef400640 1 -- 172.21.15.143:0/2977788220 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2801429172 0 0) 0x7f0cf015bf80 con 0x7f0cf0144b30 2024-01-16T15:00:06.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.863+0000 7f0cef400640 1 -- 172.21.15.143:0/2977788220 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cd0003520 con 0x7f0cf0144b30 2024-01-16T15:00:06.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.863+0000 7f0cf4e00640 1 -- 172.21.15.143:0/2977788220 >> v2:172.21.15.204:3300/0 conn(0x7f0cf015c410 msgr2=0x7f0cf015f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:06.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.863+0000 7f0cf4e00640 1 --2- 172.21.15.143:0/2977788220 >> v2:172.21.15.204:3300/0 conn(0x7f0cf015c410 0x7f0cf015f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:06.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.863+0000 7f0cf4e00640 1 -- 172.21.15.143:0/2977788220 >> v1:172.21.15.143:6789/0 conn(0x7f0cf0144b30 legacy=0x7f0cf00b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:06.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.863+0000 7f0cf4e00640 1 -- 172.21.15.143:0/2977788220 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf00b1280 con 0x7f0cf00b1de0 2024-01-16T15:00:06.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.864+0000 7f0cf4e00640 1 --2- 172.21.15.143:0/2977788220 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cf00b1de0 0x7f0cf00b2270 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0cf00cc7a0 tx=0x7f0ce802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f8aa867127863f server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:06.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.865+0000 7f0cef400640 1 -- 172.21.15.143:0/2977788220 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce802cc80 con 0x7f0cf00b1de0 2024-01-16T15:00:06.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.865+0000 7f0cef400640 1 -- 172.21.15.143:0/2977788220 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce802cde0 con 0x7f0cf00b1de0 2024-01-16T15:00:06.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.865+0000 7f0cef400640 1 -- 172.21.15.143:0/2977788220 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8034660 con 0x7f0cf00b1de0 2024-01-16T15:00:06.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.865+0000 7f0cf5800640 1 -- 172.21.15.143:0/2977788220 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cf00b1de0 msgr2=0x7f0cf00b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:06.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.865+0000 7f0cf5800640 1 --2- 172.21.15.143:0/2977788220 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cf00b1de0 0x7f0cf00b2270 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0cf00cc7a0 tx=0x7f0ce802bed0 comp rx=0 tx=0).stop 2024-01-16T15:00:06.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.865+0000 7f0cf5800640 1 -- 172.21.15.143:0/2977788220 shutdown_connections 2024-01-16T15:00:06.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.865+0000 7f0cf5800640 1 --2- 172.21.15.143:0/2977788220 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cf00b1de0 0x7f0cf00b2270 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:06.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.865+0000 7f0cf5800640 1 --2- 172.21.15.143:0/2977788220 >> v2:172.21.15.204:3300/0 conn(0x7f0cf015c410 0x7f0cf015f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:06.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.865+0000 7f0cf5800640 1 -- 172.21.15.143:0/2977788220 >> 172.21.15.143:0/2977788220 conn(0x7f0cf00cb830 msgr2=0x7f0cf00cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:06.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.865+0000 7f0cf5800640 1 -- 172.21.15.143:0/2977788220 shutdown_connections 2024-01-16T15:00:06.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.865+0000 7f0cf5800640 1 -- 172.21.15.143:0/2977788220 wait complete. 2024-01-16T15:00:06.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.867+0000 7f0cf5800640 1 Processor -- start 2024-01-16T15:00:06.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.867+0000 7f0cf5800640 1 -- start start 2024-01-16T15:00:06.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.868+0000 7f0cf5800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cf00b1de0 0x7f0cf01430b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:06.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.868+0000 7f0cf5800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cf0144b30 0x7f0cf0140100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:06.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.868+0000 7f0cf5800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0cf015c410 0x7f0cf0159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:06.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.868+0000 7f0cf5800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf015a9f0 con 0x7f0cf0144b30 2024-01-16T15:00:06.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.868+0000 7f0cf5800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf0140790 con 0x7f0cf00b1de0 2024-01-16T15:00:06.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.868+0000 7f0cf5800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf0140a70 con 0x7f0cf015c410 2024-01-16T15:00:06.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.868+0000 7f0ce7e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cf0144b30 0x7f0cf0140100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:06.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.869+0000 7f0ce7e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cf0144b30 0x7f0cf0140100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42372/0 (socket says 172.21.15.143:42372) 2024-01-16T15:00:06.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.869+0000 7f0ce7e00640 1 -- 172.21.15.143:0/2340791417 learned_addr learned my addr 172.21.15.143:0/2340791417 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:06.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.869+0000 7f0ce7e00640 1 -- 172.21.15.143:0/2340791417 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0cf015c410 msgr2=0x7f0cf0159e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:06.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.869+0000 7f0cefe00640 1 --2- 172.21.15.143:0/2340791417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cf00b1de0 0x7f0cf01430b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:06.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.869+0000 7f0cf4e00640 1 --2- 172.21.15.143:0/2340791417 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0cf015c410 0x7f0cf0159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:06.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.869+0000 7f0ce7e00640 1 --2- 172.21.15.143:0/2340791417 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0cf015c410 0x7f0cf0159e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:06.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.869+0000 7f0ce7e00640 1 -- 172.21.15.143:0/2340791417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cf00b1de0 msgr2=0x7f0cf01430b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:06.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.869+0000 7f0ce7e00640 1 --2- 172.21.15.143:0/2340791417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cf00b1de0 0x7f0cf01430b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:06.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.869+0000 7f0ce7e00640 1 -- 172.21.15.143:0/2340791417 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce8009a70 con 0x7f0cf0144b30 2024-01-16T15:00:06.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.869+0000 7f0ce7e00640 1 --2- 172.21.15.143:0/2340791417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cf0144b30 0x7f0cf0140100 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0cd800d5f0 tx=0x7f0cd800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:06.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.869+0000 7f0cecc00640 1 -- 172.21.15.143:0/2340791417 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd80125f0 con 0x7f0cf0144b30 2024-01-16T15:00:06.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.870+0000 7f0cecc00640 1 -- 172.21.15.143:0/2340791417 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd8012750 con 0x7f0cf0144b30 2024-01-16T15:00:06.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.870+0000 7f0cf5800640 1 -- 172.21.15.143:0/2340791417 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf0140d50 con 0x7f0cf0144b30 2024-01-16T15:00:06.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.870+0000 7f0cecc00640 1 -- 172.21.15.143:0/2340791417 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8012b00 con 0x7f0cf0144b30 2024-01-16T15:00:06.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.870+0000 7f0cf5800640 1 -- 172.21.15.143:0/2340791417 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf0141310 con 0x7f0cf0144b30 2024-01-16T15:00:06.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.871+0000 7f0cf5800640 1 -- 172.21.15.143:0/2340791417 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cf004e840 con 0x7f0cf0144b30 2024-01-16T15:00:06.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.872+0000 7f0cecc00640 1 -- 172.21.15.143:0/2340791417 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f0cd8012c60 con 0x7f0cf0144b30 2024-01-16T15:00:06.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.872+0000 7f0cecc00640 1 --2- 172.21.15.143:0/2340791417 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0cc806b2c0 0x7f0cc806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:06.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.873+0000 7f0cecc00640 1 -- 172.21.15.143:0/2340791417 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0cd808a030 con 0x7f0cf0144b30 2024-01-16T15:00:06.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.873+0000 7f0cefe00640 1 --2- 172.21.15.143:0/2340791417 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0cc806b2c0 0x7f0cc806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:06.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.873+0000 7f0cefe00640 1 --2- 172.21.15.143:0/2340791417 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0cc806b2c0 0x7f0cc806d780 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00011e0 tx=0x7f0ce0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:06.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:06.877+0000 7f0cecc00640 1 -- 172.21.15.143:0/2340791417 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0cd8054f30 con 0x7f0cf0144b30 2024-01-16T15:00:07.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.106+0000 7f0cf5800640 1 -- 172.21.15.143:0/2340791417 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0cf006d860 con 0x7f0cf0144b30 2024-01-16T15:00:07.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.108+0000 7f0cecc00640 1 -- 172.21.15.143:0/2340791417 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4916 (secure 0 0 0) 0x7f0cd8058550 con 0x7f0cf0144b30 2024-01-16T15:00:07.088 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:00:07.088 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3599169408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3599169408},{"type":"v1","addr":"172.21.15.204:6834","nonce":3599169408}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/859082411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":859082411},{"type":"v1","addr":"172.21.15.204:6835","nonce":859082411}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6836/1564738611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1564738611},{"type":"v1","addr":"172.21.15.143:6836","nonce":1564738611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-16T15:00:03.050082+0000","modified":"2024-01-16T15:00:06.109664+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/3120706283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6835","nonce":3120706283},{"type":"v1","addr":"172.21.15.143:6837","nonce":3120706283}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-16T15:00:07.088 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 8 2024-01-16T15:00:07.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.110+0000 7f0cf5800640 1 -- 172.21.15.143:0/2340791417 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0cc806b2c0 msgr2=0x7f0cc806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:07.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.110+0000 7f0cf5800640 1 --2- 172.21.15.143:0/2340791417 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0cc806b2c0 0x7f0cc806d780 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00011e0 tx=0x7f0ce0009040 comp rx=0 tx=0).stop 2024-01-16T15:00:07.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.110+0000 7f0cf5800640 1 -- 172.21.15.143:0/2340791417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cf0144b30 msgr2=0x7f0cf0140100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:07.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.110+0000 7f0cf5800640 1 --2- 172.21.15.143:0/2340791417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cf0144b30 0x7f0cf0140100 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0cd800d5f0 tx=0x7f0cd800dab0 comp rx=0 tx=0).stop 2024-01-16T15:00:07.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.111+0000 7f0cf5800640 1 -- 172.21.15.143:0/2340791417 shutdown_connections 2024-01-16T15:00:07.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.111+0000 7f0cf5800640 1 --2- 172.21.15.143:0/2340791417 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0cc806b2c0 0x7f0cc806d780 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:07.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.111+0000 7f0cf5800640 1 --2- 172.21.15.143:0/2340791417 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0cf015c410 0x7f0cf0159e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:07.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.111+0000 7f0cf5800640 1 --2- 172.21.15.143:0/2340791417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cf0144b30 0x7f0cf0140100 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:07.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.111+0000 7f0cf5800640 1 --2- 172.21.15.143:0/2340791417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cf00b1de0 0x7f0cf01430b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:07.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.111+0000 7f0cf5800640 1 -- 172.21.15.143:0/2340791417 >> 172.21.15.143:0/2340791417 conn(0x7f0cf00cb830 msgr2=0x7f0cf00cd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:07.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.111+0000 7f0cf5800640 1 -- 172.21.15.143:0/2340791417 shutdown_connections 2024-01-16T15:00:07.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.111+0000 7f0cf5800640 1 -- 172.21.15.143:0/2340791417 wait complete. 2024-01-16T15:00:07.109 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T15:00:07.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.335+0000 7f855d000640 1 Processor -- start 2024-01-16T15:00:07.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.335+0000 7f855d000640 1 -- start start 2024-01-16T15:00:07.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.336+0000 7f855d000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f85580b2250 0x7f855815df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:07.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.337+0000 7f855d000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f855815ab80 con 0x7f85580b2aa0 2024-01-16T15:00:07.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.337+0000 7f855d000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f855815e680 con 0x7f85580b2630 2024-01-16T15:00:07.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.337+0000 7f855d000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f855815a990 con 0x7f85580b2250 2024-01-16T15:00:07.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.337+0000 7f8555e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f85580b2250 0x7f855815df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:07.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.337+0000 7f8556800640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f85580b2aa0 0x7f855815b080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54056/0 (socket says 172.21.15.143:54056) 2024-01-16T15:00:07.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.337+0000 7f8556800640 1 -- 172.21.15.143:0/940883677 learned_addr learned my addr 172.21.15.143:0/940883677 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:07.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.337+0000 7f8555e00640 1 -- 172.21.15.143:0/940883677 >> v1:172.21.15.204:6789/0 conn(0x7f85580b2630 legacy=0x7f85580ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:07.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.337+0000 7f8555e00640 1 -- 172.21.15.143:0/940883677 >> v1:172.21.15.143:6789/0 conn(0x7f85580b2aa0 legacy=0x7f855815b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:07.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.337+0000 7f8555e00640 1 -- 172.21.15.143:0/940883677 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f855815e960 con 0x7f85580b2250 2024-01-16T15:00:07.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.338+0000 7f8555e00640 1 --2- 172.21.15.143:0/940883677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85580b2250 0x7f855815df80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f855815ad70 tx=0x7f854802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ede3a0453ed36918 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:07.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.339+0000 7f8555400640 1 -- 172.21.15.143:0/940883677 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854802d040 con 0x7f85580b2250 2024-01-16T15:00:07.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.339+0000 7f8555400640 1 -- 172.21.15.143:0/940883677 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8548031810 con 0x7f85580b2250 2024-01-16T15:00:07.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.339+0000 7f8555400640 1 -- 172.21.15.143:0/940883677 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8548031ba0 con 0x7f85580b2250 2024-01-16T15:00:07.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.339+0000 7f855d000640 1 -- 172.21.15.143:0/940883677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85580b2250 msgr2=0x7f855815df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:07.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.339+0000 7f855d000640 1 --2- 172.21.15.143:0/940883677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85580b2250 0x7f855815df80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f855815ad70 tx=0x7f854802b8b0 comp rx=0 tx=0).stop 2024-01-16T15:00:07.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.339+0000 7f855d000640 1 -- 172.21.15.143:0/940883677 shutdown_connections 2024-01-16T15:00:07.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.339+0000 7f855d000640 1 --2- 172.21.15.143:0/940883677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85580b2250 0x7f855815df80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:07.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.339+0000 7f855d000640 1 -- 172.21.15.143:0/940883677 >> 172.21.15.143:0/940883677 conn(0x7f855806a920 msgr2=0x7f855806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:07.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.340+0000 7f855d000640 1 -- 172.21.15.143:0/940883677 shutdown_connections 2024-01-16T15:00:07.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.340+0000 7f855d000640 1 -- 172.21.15.143:0/940883677 wait complete. 2024-01-16T15:00:07.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.341+0000 7f855d000640 1 Processor -- start 2024-01-16T15:00:07.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.341+0000 7f855d000640 1 -- start start 2024-01-16T15:00:07.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.342+0000 7f855d000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85580b2250 0x7f8558155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:07.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.342+0000 7f855d000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85580b2630 0x7f85581e3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:07.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.342+0000 7f855d000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85580b2aa0 0x7f8558140000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:07.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.342+0000 7f855d000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85581413b0 con 0x7f85580b2250 2024-01-16T15:00:07.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.342+0000 7f855d000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85581e42e0 con 0x7f85580b2aa0 2024-01-16T15:00:07.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.342+0000 7f855d000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8558155b30 con 0x7f85580b2630 2024-01-16T15:00:07.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.342+0000 7f8555e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85580b2250 0x7f8558155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:07.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.342+0000 7f8555e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85580b2250 0x7f8558155470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42398/0 (socket says 172.21.15.143:42398) 2024-01-16T15:00:07.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.342+0000 7f8555e00640 1 -- 172.21.15.143:0/3475477488 learned_addr learned my addr 172.21.15.143:0/3475477488 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:07.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.343+0000 7f854d200640 1 --2- 172.21.15.143:0/3475477488 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85580b2630 0x7f85581e3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:07.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.343+0000 7f8556800640 1 --2- 172.21.15.143:0/3475477488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85580b2aa0 0x7f8558140000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:07.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.343+0000 7f8555e00640 1 -- 172.21.15.143:0/3475477488 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85580b2630 msgr2=0x7f85581e3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:07.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.343+0000 7f8555e00640 1 --2- 172.21.15.143:0/3475477488 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85580b2630 0x7f85581e3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:07.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.343+0000 7f8555e00640 1 -- 172.21.15.143:0/3475477488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85580b2aa0 msgr2=0x7f8558140000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:07.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.343+0000 7f8555e00640 1 --2- 172.21.15.143:0/3475477488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85580b2aa0 0x7f8558140000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:07.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.343+0000 7f8555e00640 1 -- 172.21.15.143:0/3475477488 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8548009400 con 0x7f85580b2250 2024-01-16T15:00:07.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.343+0000 7f8556800640 1 --2- 172.21.15.143:0/3475477488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85580b2aa0 0x7f8558140000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:07.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.343+0000 7f8555e00640 1 --2- 172.21.15.143:0/3475477488 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85580b2250 0x7f8558155470 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8548009890 tx=0x7f85480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:07.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.343+0000 7f854ea00640 1 -- 172.21.15.143:0/3475477488 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854803ad60 con 0x7f85580b2250 2024-01-16T15:00:07.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.344+0000 7f854ea00640 1 -- 172.21.15.143:0/3475477488 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85480443f0 con 0x7f85580b2250 2024-01-16T15:00:07.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.344+0000 7f854ea00640 1 -- 172.21.15.143:0/3475477488 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85480447a0 con 0x7f85580b2250 2024-01-16T15:00:07.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.344+0000 7f855d000640 1 -- 172.21.15.143:0/3475477488 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85580cd7c0 con 0x7f85580b2250 2024-01-16T15:00:07.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.344+0000 7f855d000640 1 -- 172.21.15.143:0/3475477488 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8558140cf0 con 0x7f85580b2250 2024-01-16T15:00:07.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.345+0000 7f855d000640 1 -- 172.21.15.143:0/3475477488 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8558067470 con 0x7f85580b2250 2024-01-16T15:00:07.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.346+0000 7f854ea00640 1 -- 172.21.15.143:0/3475477488 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f8548044900 con 0x7f85580b2250 2024-01-16T15:00:07.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.346+0000 7f854ea00640 1 --2- 172.21.15.143:0/3475477488 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f853006b2c0 0x7f853006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:07.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.346+0000 7f854ea00640 1 -- 172.21.15.143:0/3475477488 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f85480abfe0 con 0x7f85580b2250 2024-01-16T15:00:07.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.347+0000 7f854d200640 1 --2- 172.21.15.143:0/3475477488 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f853006b2c0 0x7f853006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:07.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.347+0000 7f854d200640 1 --2- 172.21.15.143:0/3475477488 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f853006b2c0 0x7f853006d780 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f855815e030 tx=0x7f8540009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:07.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.351+0000 7f854ea00640 1 -- 172.21.15.143:0/3475477488 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8548076ee0 con 0x7f85580b2250 2024-01-16T15:00:07.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.543+0000 7f855d000640 1 -- 172.21.15.143:0/3475477488 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f85580cea00 con 0x7f85580b2250 2024-01-16T15:00:07.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.544+0000 7f854ea00640 1 -- 172.21.15.143:0/3475477488 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16565 (secure 0 0 0) 0x7f854807a500 con 0x7f85580b2250 2024-01-16T15:00:07.525 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:00:07.525 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0","created":"2024-01-16T14:59:38.532601+0000","modified":"2024-01-16T15:00:04.543620+0000","last_up_change":"2024-01-16T14:59:50.983699+0000","last_in_change":"2024-01-16T14:59:42.056297+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:59:51.727720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T15:00:01.010472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T15:00:02.041228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f4c8891d-ab29-472a-907d-8446f20f7012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6801","nonce":1569293626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6805","nonce":1569293626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6814","nonce":1569293626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6809","nonce":1569293626}]},"public_addr":"172.21.15.143:6801/1569293626","cluster_addr":"172.21.15.143:6805/1569293626","heartbeat_back_addr":"172.21.15.143:6814/1569293626","heartbeat_front_addr":"172.21.15.143:6809/1569293626","state":["exists","up"]},{"osd":1,"uuid":"c4a2788d-8b89-4992-82b2-7f4b9debd509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6804","nonce":1276841168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6808","nonce":1276841168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6813","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6815","nonce":1276841168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6811","nonce":1276841168}]},"public_addr":"172.21.15.143:6804/1276841168","cluster_addr":"172.21.15.143:6808/1276841168","heartbeat_back_addr":"172.21.15.143:6815/1276841168","heartbeat_front_addr":"172.21.15.143:6811/1276841168","state":["exists","up"]},{"osd":2,"uuid":"64645c1d-31b2-4794-b0f2-7ccf8df30393","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6817","nonce":1217670170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6819","nonce":1217670170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6823","nonce":1217670170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6821","nonce":1217670170}]},"public_addr":"172.21.15.143:6817/1217670170","cluster_addr":"172.21.15.143:6819/1217670170","heartbeat_back_addr":"172.21.15.143:6823/1217670170","heartbeat_front_addr":"172.21.15.143:6821/1217670170","state":["exists","up"]},{"osd":3,"uuid":"73674129-a84e-4d31-8247-7d0179ae6df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6825","nonce":3414188012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6827","nonce":3414188012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6831","nonce":3414188012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6829","nonce":3414188012}]},"public_addr":"172.21.15.143:6825/3414188012","cluster_addr":"172.21.15.143:6827/3414188012","heartbeat_back_addr":"172.21.15.143:6831/3414188012","heartbeat_front_addr":"172.21.15.143:6829/3414188012","state":["exists","up"]},{"osd":4,"uuid":"cea28c9b-2f1c-4087-b415-9960789131d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6811","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6812","nonce":1070255716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6816","nonce":1070255716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6823","nonce":1070255716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6820","nonce":1070255716}]},"public_addr":"172.21.15.204:6812/1070255716","cluster_addr":"172.21.15.204:6816/1070255716","heartbeat_back_addr":"172.21.15.204:6823/1070255716","heartbeat_front_addr":"172.21.15.204:6820/1070255716","state":["exists","up"]},{"osd":5,"uuid":"33b63560-5574-4e78-983f-b398d67a2cb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6809","nonce":3605417837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6813","nonce":3605417837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6819","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6821","nonce":3605417837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6817","nonce":3605417837}]},"public_addr":"172.21.15.204:6809/3605417837","cluster_addr":"172.21.15.204:6813/3605417837","heartbeat_back_addr":"172.21.15.204:6821/3605417837","heartbeat_front_addr":"172.21.15.204:6817/3605417837","state":["exists","up"]},{"osd":6,"uuid":"df397427-8940-4eee-9c79-eece669d735d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6801","nonce":1085000534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6803","nonce":1085000534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6808","nonce":1085000534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6805","nonce":1085000534}]},"public_addr":"172.21.15.204:6801/1085000534","cluster_addr":"172.21.15.204:6803/1085000534","heartbeat_back_addr":"172.21.15.204:6808/1085000534","heartbeat_front_addr":"172.21.15.204:6805/1085000534","state":["exists","up"]},{"osd":7,"uuid":"a2e5a15c-8312-405a-ac02-f7e1fc9df2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6825","nonce":1337220609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6827","nonce":1337220609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6831","nonce":1337220609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6829","nonce":1337220609}]},"public_addr":"172.21.15.204:6825/1337220609","cluster_addr":"172.21.15.204:6827/1337220609","heartbeat_back_addr":"172.21.15.204:6831/1337220609","heartbeat_front_addr":"172.21.15.204:6829/1337220609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:49.984507+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.395306+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.632998+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.373627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.342920+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.438366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.331926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.406657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T15:00:07.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.547+0000 7f855d000640 1 -- 172.21.15.143:0/3475477488 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f853006b2c0 msgr2=0x7f853006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:07.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.547+0000 7f855d000640 1 --2- 172.21.15.143:0/3475477488 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f853006b2c0 0x7f853006d780 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f855815e030 tx=0x7f8540009040 comp rx=0 tx=0).stop 2024-01-16T15:00:07.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.547+0000 7f855d000640 1 -- 172.21.15.143:0/3475477488 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85580b2250 msgr2=0x7f8558155470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:07.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.547+0000 7f855d000640 1 --2- 172.21.15.143:0/3475477488 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85580b2250 0x7f8558155470 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8548009890 tx=0x7f85480046c0 comp rx=0 tx=0).stop 2024-01-16T15:00:07.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.548+0000 7f855d000640 1 -- 172.21.15.143:0/3475477488 shutdown_connections 2024-01-16T15:00:07.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.548+0000 7f855d000640 1 --2- 172.21.15.143:0/3475477488 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f853006b2c0 0x7f853006d780 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:07.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.548+0000 7f855d000640 1 --2- 172.21.15.143:0/3475477488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85580b2aa0 0x7f8558140000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:07.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.548+0000 7f855d000640 1 --2- 172.21.15.143:0/3475477488 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85580b2630 0x7f85581e3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:07.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.548+0000 7f855d000640 1 --2- 172.21.15.143:0/3475477488 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85580b2250 0x7f8558155470 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:07.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.548+0000 7f855d000640 1 -- 172.21.15.143:0/3475477488 >> 172.21.15.143:0/3475477488 conn(0x7f855806a920 msgr2=0x7f85581592a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:07.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.548+0000 7f855d000640 1 -- 172.21.15.143:0/3475477488 shutdown_connections 2024-01-16T15:00:07.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.548+0000 7f855d000640 1 -- 172.21.15.143:0/3475477488 wait complete. 2024-01-16T15:00:07.549 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-16T15:00:07.549 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-16T15:00:07.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.767+0000 7f906b200640 1 Processor -- start 2024-01-16T15:00:07.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.768+0000 7f906b200640 1 -- start start 2024-01-16T15:00:07.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.769+0000 7f906b200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f906415b080 0x7f906415b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:07.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.769+0000 7f906b200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f906415e3f0 con 0x7f906415df80 2024-01-16T15:00:07.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.769+0000 7f906b200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f906415ab80 con 0x7f90640b2250 2024-01-16T15:00:07.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.769+0000 7f906b200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f906415a990 con 0x7f906415b080 2024-01-16T15:00:07.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.769+0000 7f905b200640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f906415df80 0x7f90640ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54066/0 (socket says 172.21.15.143:54066) 2024-01-16T15:00:07.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.769+0000 7f905b200640 1 -- 172.21.15.143:0/2019047558 learned_addr learned my addr 172.21.15.143:0/2019047558 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:07.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.770+0000 7f9069200640 1 --2- 172.21.15.143:0/2019047558 >> v2:172.21.15.204:3300/0 conn(0x7f906415b080 0x7f906415b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:07.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.770+0000 7f9063400640 1 -- 172.21.15.143:0/2019047558 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3016092211 0 0) 0x7f906415e3f0 con 0x7f906415df80 2024-01-16T15:00:07.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.770+0000 7f9063400640 1 -- 172.21.15.143:0/2019047558 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9044005730 con 0x7f906415df80 2024-01-16T15:00:07.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.770+0000 7f9063400640 1 -- 172.21.15.143:0/2019047558 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4271964681 0 0) 0x7f9044005730 con 0x7f906415df80 2024-01-16T15:00:07.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.770+0000 7f9063400640 1 -- 172.21.15.143:0/2019047558 >> v2:172.21.15.204:3300/0 conn(0x7f906415b080 msgr2=0x7f906415b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:07.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.770+0000 7f9063400640 1 --2- 172.21.15.143:0/2019047558 >> v2:172.21.15.204:3300/0 conn(0x7f906415b080 0x7f906415b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:07.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.770+0000 7f9063400640 1 -- 172.21.15.143:0/2019047558 >> v1:172.21.15.204:6789/0 conn(0x7f90640b2250 legacy=0x7f90640b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:07.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.770+0000 7f9063400640 1 -- 172.21.15.143:0/2019047558 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90640b2d40 con 0x7f906415df80 2024-01-16T15:00:07.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.770+0000 7f9063400640 1 -- 172.21.15.143:0/2019047558 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7f904c001670 con 0x7f906415df80 2024-01-16T15:00:07.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.770+0000 7f9063400640 1 -- 172.21.15.143:0/2019047558 >> v1:172.21.15.143:6789/0 conn(0x7f906415df80 legacy=0x7f90640ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:07.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.770+0000 7f9063400640 1 --2- 172.21.15.143:0/2019047558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9044005b80 0x7f9044005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:07.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.770+0000 7f9063400640 1 --2- 172.21.15.143:0/2019047558 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90440069d0 0x7f9044006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:07.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.770+0000 7f9063400640 1 --2- 172.21.15.143:0/2019047558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9044007390 0x7f904400db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:07.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.771+0000 7f9063400640 1 -- 172.21.15.143:0/2019047558 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904400e200 con 0x7f9044005b80 2024-01-16T15:00:07.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.771+0000 7f9063400640 1 -- 172.21.15.143:0/2019047558 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904400e370 con 0x7f9044007390 2024-01-16T15:00:07.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.771+0000 7f9063400640 1 -- 172.21.15.143:0/2019047558 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f904400e530 con 0x7f90440069d0 2024-01-16T15:00:07.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.771+0000 7f9063e00640 1 --2- 172.21.15.143:0/2019047558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9044005b80 0x7f9044005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:07.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.771+0000 7f905b200640 1 --2- 172.21.15.143:0/2019047558 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90440069d0 0x7f9044006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:07.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.771+0000 7f9063e00640 1 -- 172.21.15.143:0/2019047558 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90440069d0 msgr2=0x7f9044006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:07.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.771+0000 7f9063e00640 1 --2- 172.21.15.143:0/2019047558 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90440069d0 0x7f9044006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:07.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.771+0000 7f9063e00640 1 -- 172.21.15.143:0/2019047558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9044007390 msgr2=0x7f904400db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:07.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.771+0000 7f9063e00640 1 --2- 172.21.15.143:0/2019047558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9044007390 0x7f904400db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:07.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.771+0000 7f9063e00640 1 -- 172.21.15.143:0/2019047558 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9044005730 con 0x7f9044005b80 2024-01-16T15:00:07.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.771+0000 7f9069200640 1 -- 172.21.15.143:0/2019047558 reap_dead start 2024-01-16T15:00:07.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.771+0000 7f905b200640 1 --2- 172.21.15.143:0/2019047558 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90440069d0 0x7f9044006e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:07.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.772+0000 7f9063e00640 1 --2- 172.21.15.143:0/2019047558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9044005b80 0x7f9044005fa0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9054001750 tx=0x7f9044007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7b24ea1f7cafe28 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:07.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.772+0000 7f9063400640 1 -- 172.21.15.143:0/2019047558 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90540052d0 con 0x7f9044005b80 2024-01-16T15:00:07.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.772+0000 7f9063400640 1 -- 172.21.15.143:0/2019047558 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f905402e050 con 0x7f9044005b80 2024-01-16T15:00:07.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.772+0000 7f9063400640 1 -- 172.21.15.143:0/2019047558 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9054003830 con 0x7f9044005b80 2024-01-16T15:00:07.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.773+0000 7f906b200640 1 -- 172.21.15.143:0/2019047558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9044005b80 msgr2=0x7f9044005fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:07.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.773+0000 7f906b200640 1 --2- 172.21.15.143:0/2019047558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9044005b80 0x7f9044005fa0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9054001750 tx=0x7f9044007a00 comp rx=0 tx=0).stop 2024-01-16T15:00:07.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.773+0000 7f906b200640 1 -- 172.21.15.143:0/2019047558 shutdown_connections 2024-01-16T15:00:07.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.773+0000 7f906b200640 1 --2- 172.21.15.143:0/2019047558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9044005b80 0x7f9044005fa0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:07.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.773+0000 7f906b200640 1 -- 172.21.15.143:0/2019047558 >> 172.21.15.143:0/2019047558 conn(0x7f906406a920 msgr2=0x7f906406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:07.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.773+0000 7f906b200640 1 -- 172.21.15.143:0/2019047558 shutdown_connections 2024-01-16T15:00:07.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.773+0000 7f906b200640 1 -- 172.21.15.143:0/2019047558 wait complete. 2024-01-16T15:00:07.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.775+0000 7f906b200640 1 Processor -- start 2024-01-16T15:00:07.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.775+0000 7f906b200640 1 -- start start 2024-01-16T15:00:07.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.776+0000 7f906b200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9044005b80 0x7f9064148110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:07.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.776+0000 7f906b200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90640ca980 0x7f90640adca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:07.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.776+0000 7f906b200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90640ae1e0 0x7f906415bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:07.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.776+0000 7f906b200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90640c9940 con 0x7f90640ca980 2024-01-16T15:00:07.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.776+0000 7f906b200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90640c9bc0 con 0x7f90640ae1e0 2024-01-16T15:00:07.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.776+0000 7f906b200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90640c9ea0 con 0x7f9044005b80 2024-01-16T15:00:07.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.776+0000 7f905b200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90640ca980 0x7f90640adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:07.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.776+0000 7f905b200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90640ca980 0x7f90640adca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42418/0 (socket says 172.21.15.143:42418) 2024-01-16T15:00:07.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.776+0000 7f905b200640 1 -- 172.21.15.143:0/2744541010 learned_addr learned my addr 172.21.15.143:0/2744541010 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:07.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.776+0000 7f9063e00640 1 --2- 172.21.15.143:0/2744541010 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9044005b80 0x7f9064148110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:07.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.777+0000 7f9069200640 1 --2- 172.21.15.143:0/2744541010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90640ae1e0 0x7f906415bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:07.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.777+0000 7f905b200640 1 -- 172.21.15.143:0/2744541010 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9044005b80 msgr2=0x7f9064148110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:07.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.777+0000 7f905b200640 1 --2- 172.21.15.143:0/2744541010 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9044005b80 0x7f9064148110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:07.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.777+0000 7f905b200640 1 -- 172.21.15.143:0/2744541010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90640ae1e0 msgr2=0x7f906415bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:07.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.777+0000 7f905b200640 1 --2- 172.21.15.143:0/2744541010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90640ae1e0 0x7f906415bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:07.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.777+0000 7f905b200640 1 -- 172.21.15.143:0/2744541010 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90640ca180 con 0x7f90640ca980 2024-01-16T15:00:07.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.777+0000 7f9069200640 1 --2- 172.21.15.143:0/2744541010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90640ae1e0 0x7f906415bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:07.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.777+0000 7f905b200640 1 --2- 172.21.15.143:0/2744541010 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90640ca980 0x7f90640adca0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f904c00d7a0 tx=0x7f904c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:07.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.777+0000 7f9060c00640 1 -- 172.21.15.143:0/2744541010 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c0074e0 con 0x7f90640ca980 2024-01-16T15:00:07.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.777+0000 7f9060c00640 1 -- 172.21.15.143:0/2744541010 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904c007640 con 0x7f90640ca980 2024-01-16T15:00:07.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.777+0000 7f906b200640 1 -- 172.21.15.143:0/2744541010 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90640ca460 con 0x7f90640ca980 2024-01-16T15:00:07.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.777+0000 7f9060c00640 1 -- 172.21.15.143:0/2744541010 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c004a00 con 0x7f90640ca980 2024-01-16T15:00:07.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.777+0000 7f906b200640 1 -- 172.21.15.143:0/2744541010 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90641488e0 con 0x7f90640ca980 2024-01-16T15:00:07.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.779+0000 7f906b200640 1 -- 172.21.15.143:0/2744541010 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9064067470 con 0x7f90640ca980 2024-01-16T15:00:07.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.779+0000 7f9060c00640 1 -- 172.21.15.143:0/2744541010 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f904c004b60 con 0x7f90640ca980 2024-01-16T15:00:07.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.780+0000 7f9060c00640 1 --2- 172.21.15.143:0/2744541010 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f903c06b2c0 0x7f903c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:07.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.780+0000 7f9060c00640 1 -- 172.21.15.143:0/2744541010 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f904c0897b0 con 0x7f90640ca980 2024-01-16T15:00:07.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.781+0000 7f9063e00640 1 --2- 172.21.15.143:0/2744541010 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f903c06b2c0 0x7f903c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:07.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.781+0000 7f9063e00640 1 --2- 172.21.15.143:0/2744541010 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f903c06b2c0 0x7f903c06d780 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9054033000 tx=0x7f9054009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:07.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.785+0000 7f9060c00640 1 -- 172.21.15.143:0/2744541010 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f904c0546b0 con 0x7f90640ca980 2024-01-16T15:00:07.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.961+0000 7f906b200640 1 -- 172.21.15.143:0/2744541010 --> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f906415ec40 con 0x7f903c06b2c0 2024-01-16T15:00:07.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.976+0000 7f9060c00640 1 -- 172.21.15.143:0/2744541010 <== mgr.4113 v2:172.21.15.143:6832/3153492115 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264522 (secure 0 0 0) 0x7f9054005490 con 0x7f903c06b2c0 2024-01-16T15:00:07.960 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:00:07.961 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-16T15:00:07.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.982+0000 7f906b200640 1 -- 172.21.15.143:0/2744541010 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f903c06b2c0 msgr2=0x7f903c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:07.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.982+0000 7f906b200640 1 --2- 172.21.15.143:0/2744541010 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f903c06b2c0 0x7f903c06d780 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9054033000 tx=0x7f9054009030 comp rx=0 tx=0).stop 2024-01-16T15:00:07.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.982+0000 7f906b200640 1 -- 172.21.15.143:0/2744541010 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90640ca980 msgr2=0x7f90640adca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:07.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.982+0000 7f906b200640 1 --2- 172.21.15.143:0/2744541010 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90640ca980 0x7f90640adca0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f904c00d7a0 tx=0x7f904c00dc60 comp rx=0 tx=0).stop 2024-01-16T15:00:07.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.982+0000 7f906b200640 1 -- 172.21.15.143:0/2744541010 shutdown_connections 2024-01-16T15:00:07.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.982+0000 7f906b200640 1 --2- 172.21.15.143:0/2744541010 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f903c06b2c0 0x7f903c06d780 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:07.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.982+0000 7f906b200640 1 --2- 172.21.15.143:0/2744541010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90640ae1e0 0x7f906415bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:07.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.982+0000 7f906b200640 1 --2- 172.21.15.143:0/2744541010 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90640ca980 0x7f90640adca0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:07.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.982+0000 7f906b200640 1 --2- 172.21.15.143:0/2744541010 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9044005b80 0x7f9064148110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:07.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.982+0000 7f906b200640 1 -- 172.21.15.143:0/2744541010 >> 172.21.15.143:0/2744541010 conn(0x7f906406a920 msgr2=0x7f9064158fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:07.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.982+0000 7f906b200640 1 -- 172.21.15.143:0/2744541010 shutdown_connections 2024-01-16T15:00:07.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:07.982+0000 7f906b200640 1 -- 172.21.15.143:0/2744541010 wait complete. 2024-01-16T15:00:07.979 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-01-16T15:00:07.653854+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":188,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219004,"kb_used_data":2820,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755716,"statfs":{"total":773094113280,"available":772869853184,"internally_reserved":0,"allocated":2887680,"data_stored":1751817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":58,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001036"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.558033+0000","last_change":"2024-01-16T15:00:03.579227+0000","last_active":"2024-01-16T15:00:04.558033+0000","last_peered":"2024-01-16T15:00:04.558033+0000","last_clean":"2024-01-16T15:00:04.558033+0000","last_became_active":"2024-01-16T15:00:03.578921+0000","last_became_peered":"2024-01-16T15:00:03.578921+0000","last_unstale":"2024-01-16T15:00:04.558033+0000","last_undegraded":"2024-01-16T15:00:04.558033+0000","last_fullsized":"2024-01-16T15:00:04.558033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:32:18.387639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.554616+0000","last_change":"2024-01-16T15:00:03.571130+0000","last_active":"2024-01-16T15:00:04.554616+0000","last_peered":"2024-01-16T15:00:04.554616+0000","last_clean":"2024-01-16T15:00:04.554616+0000","last_became_active":"2024-01-16T15:00:03.570816+0000","last_became_peered":"2024-01-16T15:00:03.570816+0000","last_unstale":"2024-01-16T15:00:04.554616+0000","last_undegraded":"2024-01-16T15:00:04.554616+0000","last_fullsized":"2024-01-16T15:00:04.554616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:06:24.858403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323971+0000","last_change":"2024-01-16T15:00:03.578171+0000","last_active":"2024-01-16T15:00:05.323971+0000","last_peered":"2024-01-16T15:00:05.323971+0000","last_clean":"2024-01-16T15:00:05.323971+0000","last_became_active":"2024-01-16T15:00:03.577849+0000","last_became_peered":"2024-01-16T15:00:03.577849+0000","last_unstale":"2024-01-16T15:00:05.323971+0000","last_undegraded":"2024-01-16T15:00:05.323971+0000","last_fullsized":"2024-01-16T15:00:05.323971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:11:11.328141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323719+0000","last_change":"2024-01-16T15:00:03.585019+0000","last_active":"2024-01-16T15:00:05.323719+0000","last_peered":"2024-01-16T15:00:05.323719+0000","last_clean":"2024-01-16T15:00:05.323719+0000","last_became_active":"2024-01-16T15:00:03.584747+0000","last_became_peered":"2024-01-16T15:00:03.584747+0000","last_unstale":"2024-01-16T15:00:05.323719+0000","last_undegraded":"2024-01-16T15:00:05.323719+0000","last_fullsized":"2024-01-16T15:00:05.323719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:44:37.235476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.556701+0000","last_change":"2024-01-16T15:00:03.577404+0000","last_active":"2024-01-16T15:00:04.556701+0000","last_peered":"2024-01-16T15:00:04.556701+0000","last_clean":"2024-01-16T15:00:04.556701+0000","last_became_active":"2024-01-16T15:00:03.577051+0000","last_became_peered":"2024-01-16T15:00:03.577051+0000","last_unstale":"2024-01-16T15:00:04.556701+0000","last_undegraded":"2024-01-16T15:00:04.556701+0000","last_fullsized":"2024-01-16T15:00:04.556701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:44:37.235476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.322548+0000","last_change":"2024-01-16T15:00:03.589065+0000","last_active":"2024-01-16T15:00:05.322548+0000","last_peered":"2024-01-16T15:00:05.322548+0000","last_clean":"2024-01-16T15:00:05.322548+0000","last_became_active":"2024-01-16T15:00:03.588776+0000","last_became_peered":"2024-01-16T15:00:03.588776+0000","last_unstale":"2024-01-16T15:00:05.322548+0000","last_undegraded":"2024-01-16T15:00:05.322548+0000","last_fullsized":"2024-01-16T15:00:05.322548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:06:24.858403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.556141+0000","last_change":"2024-01-16T15:00:03.570611+0000","last_active":"2024-01-16T15:00:04.556141+0000","last_peered":"2024-01-16T15:00:04.556141+0000","last_clean":"2024-01-16T15:00:04.556141+0000","last_became_active":"2024-01-16T15:00:03.569839+0000","last_became_peered":"2024-01-16T15:00:03.569839+0000","last_unstale":"2024-01-16T15:00:04.556141+0000","last_undegraded":"2024-01-16T15:00:04.556141+0000","last_fullsized":"2024-01-16T15:00:04.556141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:09:47.428018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.127040+0000","last_change":"2024-01-16T15:00:03.586954+0000","last_active":"2024-01-16T15:00:05.127040+0000","last_peered":"2024-01-16T15:00:05.127040+0000","last_clean":"2024-01-16T15:00:05.127040+0000","last_became_active":"2024-01-16T15:00:03.586663+0000","last_became_peered":"2024-01-16T15:00:03.586663+0000","last_unstale":"2024-01-16T15:00:05.127040+0000","last_undegraded":"2024-01-16T15:00:05.127040+0000","last_fullsized":"2024-01-16T15:00:05.127040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:34:39.229613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.558126+0000","last_change":"2024-01-16T15:00:03.596120+0000","last_active":"2024-01-16T15:00:04.558126+0000","last_peered":"2024-01-16T15:00:04.558126+0000","last_clean":"2024-01-16T15:00:04.558126+0000","last_became_active":"2024-01-16T15:00:03.595866+0000","last_became_peered":"2024-01-16T15:00:03.595866+0000","last_unstale":"2024-01-16T15:00:04.558126+0000","last_undegraded":"2024-01-16T15:00:04.558126+0000","last_fullsized":"2024-01-16T15:00:04.558126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:53:57.952288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.750726+0000","last_change":"2024-01-16T15:00:03.588019+0000","last_active":"2024-01-16T15:00:04.750726+0000","last_peered":"2024-01-16T15:00:04.750726+0000","last_clean":"2024-01-16T15:00:04.750726+0000","last_became_active":"2024-01-16T15:00:03.587359+0000","last_became_peered":"2024-01-16T15:00:03.587359+0000","last_unstale":"2024-01-16T15:00:04.750726+0000","last_undegraded":"2024-01-16T15:00:04.750726+0000","last_fullsized":"2024-01-16T15:00:04.750726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:31:56.335251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.750616+0000","last_change":"2024-01-16T15:00:03.590950+0000","last_active":"2024-01-16T15:00:04.750616+0000","last_peered":"2024-01-16T15:00:04.750616+0000","last_clean":"2024-01-16T15:00:04.750616+0000","last_became_active":"2024-01-16T15:00:03.590702+0000","last_became_peered":"2024-01-16T15:00:03.590702+0000","last_unstale":"2024-01-16T15:00:04.750616+0000","last_undegraded":"2024-01-16T15:00:04.750616+0000","last_fullsized":"2024-01-16T15:00:04.750616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:04:13.308774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.555500+0000","last_change":"2024-01-16T15:00:03.582929+0000","last_active":"2024-01-16T15:00:04.555500+0000","last_peered":"2024-01-16T15:00:04.555500+0000","last_clean":"2024-01-16T15:00:04.555500+0000","last_became_active":"2024-01-16T15:00:03.582040+0000","last_became_peered":"2024-01-16T15:00:03.582040+0000","last_unstale":"2024-01-16T15:00:04.555500+0000","last_undegraded":"2024-01-16T15:00:04.555500+0000","last_fullsized":"2024-01-16T15:00:04.555500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:10:55.291745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.127726+0000","last_change":"2024-01-16T15:00:03.581183+0000","last_active":"2024-01-16T15:00:05.127726+0000","last_peered":"2024-01-16T15:00:05.127726+0000","last_clean":"2024-01-16T15:00:05.127726+0000","last_became_active":"2024-01-16T15:00:03.578642+0000","last_became_peered":"2024-01-16T15:00:03.578642+0000","last_unstale":"2024-01-16T15:00:05.127726+0000","last_undegraded":"2024-01-16T15:00:05.127726+0000","last_fullsized":"2024-01-16T15:00:05.127726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:06:24.858403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323409+0000","last_change":"2024-01-16T15:00:03.570282+0000","last_active":"2024-01-16T15:00:05.323409+0000","last_peered":"2024-01-16T15:00:05.323409+0000","last_clean":"2024-01-16T15:00:05.323409+0000","last_became_active":"2024-01-16T15:00:03.569988+0000","last_became_peered":"2024-01-16T15:00:03.569988+0000","last_unstale":"2024-01-16T15:00:05.323409+0000","last_undegraded":"2024-01-16T15:00:05.323409+0000","last_fullsized":"2024-01-16T15:00:05.323409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:16:00.005822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.554056+0000","last_change":"2024-01-16T15:00:03.584705+0000","last_active":"2024-01-16T15:00:04.554056+0000","last_peered":"2024-01-16T15:00:04.554056+0000","last_clean":"2024-01-16T15:00:04.554056+0000","last_became_active":"2024-01-16T15:00:03.584361+0000","last_became_peered":"2024-01-16T15:00:03.584361+0000","last_unstale":"2024-01-16T15:00:04.554056+0000","last_undegraded":"2024-01-16T15:00:04.554056+0000","last_fullsized":"2024-01-16T15:00:04.554056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:11:11.328141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.322398+0000","last_change":"2024-01-16T15:00:03.587177+0000","last_active":"2024-01-16T15:00:05.322398+0000","last_peered":"2024-01-16T15:00:05.322398+0000","last_clean":"2024-01-16T15:00:05.322398+0000","last_became_active":"2024-01-16T15:00:03.586803+0000","last_became_peered":"2024-01-16T15:00:03.586803+0000","last_unstale":"2024-01-16T15:00:05.322398+0000","last_undegraded":"2024-01-16T15:00:05.322398+0000","last_fullsized":"2024-01-16T15:00:05.322398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:03:10.226190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323936+0000","last_change":"2024-01-16T15:00:03.595310+0000","last_active":"2024-01-16T15:00:05.323936+0000","last_peered":"2024-01-16T15:00:05.323936+0000","last_clean":"2024-01-16T15:00:05.323936+0000","last_became_active":"2024-01-16T15:00:03.589586+0000","last_became_peered":"2024-01-16T15:00:03.589586+0000","last_unstale":"2024-01-16T15:00:05.323936+0000","last_undegraded":"2024-01-16T15:00:05.323936+0000","last_fullsized":"2024-01-16T15:00:05.323936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:29:26.138536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.322465+0000","last_change":"2024-01-16T15:00:03.583181+0000","last_active":"2024-01-16T15:00:05.322465+0000","last_peered":"2024-01-16T15:00:05.322465+0000","last_clean":"2024-01-16T15:00:05.322465+0000","last_became_active":"2024-01-16T15:00:03.582404+0000","last_became_peered":"2024-01-16T15:00:03.582404+0000","last_unstale":"2024-01-16T15:00:05.322465+0000","last_undegraded":"2024-01-16T15:00:05.322465+0000","last_fullsized":"2024-01-16T15:00:05.322465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:25:28.863720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.557296+0000","last_change":"2024-01-16T15:00:03.577376+0000","last_active":"2024-01-16T15:00:04.557296+0000","last_peered":"2024-01-16T15:00:04.557296+0000","last_clean":"2024-01-16T15:00:04.557296+0000","last_became_active":"2024-01-16T15:00:03.577167+0000","last_became_peered":"2024-01-16T15:00:03.577167+0000","last_unstale":"2024-01-16T15:00:04.557296+0000","last_undegraded":"2024-01-16T15:00:04.557296+0000","last_fullsized":"2024-01-16T15:00:04.557296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:14:46.103689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.750577+0000","last_change":"2024-01-16T15:00:03.587122+0000","last_active":"2024-01-16T15:00:04.750577+0000","last_peered":"2024-01-16T15:00:04.750577+0000","last_clean":"2024-01-16T15:00:04.750577+0000","last_became_active":"2024-01-16T15:00:03.586244+0000","last_became_peered":"2024-01-16T15:00:03.586244+0000","last_unstale":"2024-01-16T15:00:04.750577+0000","last_undegraded":"2024-01-16T15:00:04.750577+0000","last_fullsized":"2024-01-16T15:00:04.750577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:57:14.928029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323320+0000","last_change":"2024-01-16T15:00:03.572186+0000","last_active":"2024-01-16T15:00:05.323320+0000","last_peered":"2024-01-16T15:00:05.323320+0000","last_clean":"2024-01-16T15:00:05.323320+0000","last_became_active":"2024-01-16T15:00:03.571795+0000","last_became_peered":"2024-01-16T15:00:03.571795+0000","last_unstale":"2024-01-16T15:00:05.323320+0000","last_undegraded":"2024-01-16T15:00:05.323320+0000","last_fullsized":"2024-01-16T15:00:05.323320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:18:11.185813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.322924+0000","last_change":"2024-01-16T15:00:03.589661+0000","last_active":"2024-01-16T15:00:05.322924+0000","last_peered":"2024-01-16T15:00:05.322924+0000","last_clean":"2024-01-16T15:00:05.322924+0000","last_became_active":"2024-01-16T15:00:03.589365+0000","last_became_peered":"2024-01-16T15:00:03.589365+0000","last_unstale":"2024-01-16T15:00:05.322924+0000","last_undegraded":"2024-01-16T15:00:05.322924+0000","last_fullsized":"2024-01-16T15:00:05.322924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:04:13.308774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.127047+0000","last_change":"2024-01-16T15:00:03.591522+0000","last_active":"2024-01-16T15:00:05.127047+0000","last_peered":"2024-01-16T15:00:05.127047+0000","last_clean":"2024-01-16T15:00:05.127047+0000","last_became_active":"2024-01-16T15:00:03.591143+0000","last_became_peered":"2024-01-16T15:00:03.591143+0000","last_unstale":"2024-01-16T15:00:05.127047+0000","last_undegraded":"2024-01-16T15:00:05.127047+0000","last_fullsized":"2024-01-16T15:00:05.127047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:11:11.328141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.322833+0000","last_change":"2024-01-16T15:00:03.585381+0000","last_active":"2024-01-16T15:00:05.322833+0000","last_peered":"2024-01-16T15:00:05.322833+0000","last_clean":"2024-01-16T15:00:05.322833+0000","last_became_active":"2024-01-16T15:00:03.584858+0000","last_became_peered":"2024-01-16T15:00:03.584858+0000","last_unstale":"2024-01-16T15:00:05.322833+0000","last_undegraded":"2024-01-16T15:00:05.322833+0000","last_fullsized":"2024-01-16T15:00:05.322833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:57:14.928029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.127981+0000","last_change":"2024-01-16T15:00:03.591627+0000","last_active":"2024-01-16T15:00:05.127981+0000","last_peered":"2024-01-16T15:00:05.127981+0000","last_clean":"2024-01-16T15:00:05.127981+0000","last_became_active":"2024-01-16T15:00:03.591216+0000","last_became_peered":"2024-01-16T15:00:03.591216+0000","last_unstale":"2024-01-16T15:00:05.127981+0000","last_undegraded":"2024-01-16T15:00:05.127981+0000","last_fullsized":"2024-01-16T15:00:05.127981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:31:56.335251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.750822+0000","last_change":"2024-01-16T15:00:03.588262+0000","last_active":"2024-01-16T15:00:04.750822+0000","last_peered":"2024-01-16T15:00:04.750822+0000","last_clean":"2024-01-16T15:00:04.750822+0000","last_became_active":"2024-01-16T15:00:03.587728+0000","last_became_peered":"2024-01-16T15:00:03.587728+0000","last_unstale":"2024-01-16T15:00:04.750822+0000","last_undegraded":"2024-01-16T15:00:04.750822+0000","last_fullsized":"2024-01-16T15:00:04.750822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:05:08.127179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.554267+0000","last_change":"2024-01-16T15:00:03.576892+0000","last_active":"2024-01-16T15:00:04.554267+0000","last_peered":"2024-01-16T15:00:04.554267+0000","last_clean":"2024-01-16T15:00:04.554267+0000","last_became_active":"2024-01-16T15:00:03.574566+0000","last_became_peered":"2024-01-16T15:00:03.574566+0000","last_unstale":"2024-01-16T15:00:04.554267+0000","last_undegraded":"2024-01-16T15:00:04.554267+0000","last_fullsized":"2024-01-16T15:00:04.554267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:33:59.253631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.127898+0000","last_change":"2024-01-16T15:00:03.591601+0000","last_active":"2024-01-16T15:00:05.127898+0000","last_peered":"2024-01-16T15:00:05.127898+0000","last_clean":"2024-01-16T15:00:05.127898+0000","last_became_active":"2024-01-16T15:00:03.591271+0000","last_became_peered":"2024-01-16T15:00:03.591271+0000","last_unstale":"2024-01-16T15:00:05.127898+0000","last_undegraded":"2024-01-16T15:00:05.127898+0000","last_fullsized":"2024-01-16T15:00:05.127898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:04:13.308774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323503+0000","last_change":"2024-01-16T15:00:03.580655+0000","last_active":"2024-01-16T15:00:05.323503+0000","last_peered":"2024-01-16T15:00:05.323503+0000","last_clean":"2024-01-16T15:00:05.323503+0000","last_became_active":"2024-01-16T15:00:03.580091+0000","last_became_peered":"2024-01-16T15:00:03.580091+0000","last_unstale":"2024-01-16T15:00:05.323503+0000","last_undegraded":"2024-01-16T15:00:05.323503+0000","last_fullsized":"2024-01-16T15:00:05.323503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:34:39.229613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.322959+0000","last_change":"2024-01-16T15:00:03.588486+0000","last_active":"2024-01-16T15:00:05.322959+0000","last_peered":"2024-01-16T15:00:05.322959+0000","last_clean":"2024-01-16T15:00:05.322959+0000","last_became_active":"2024-01-16T15:00:03.586447+0000","last_became_peered":"2024-01-16T15:00:03.586447+0000","last_unstale":"2024-01-16T15:00:05.322959+0000","last_undegraded":"2024-01-16T15:00:05.322959+0000","last_fullsized":"2024-01-16T15:00:05.322959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:31:56.335251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.322016+0000","last_change":"2024-01-16T15:00:03.581151+0000","last_active":"2024-01-16T15:00:05.322016+0000","last_peered":"2024-01-16T15:00:05.322016+0000","last_clean":"2024-01-16T15:00:05.322016+0000","last_became_active":"2024-01-16T15:00:03.579463+0000","last_became_peered":"2024-01-16T15:00:03.579463+0000","last_unstale":"2024-01-16T15:00:05.322016+0000","last_undegraded":"2024-01-16T15:00:05.322016+0000","last_fullsized":"2024-01-16T15:00:05.322016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:10:55.291745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.750064+0000","last_change":"2024-01-16T15:00:03.588682+0000","last_active":"2024-01-16T15:00:04.750064+0000","last_peered":"2024-01-16T15:00:04.750064+0000","last_clean":"2024-01-16T15:00:04.750064+0000","last_became_active":"2024-01-16T15:00:03.588148+0000","last_became_peered":"2024-01-16T15:00:03.588148+0000","last_unstale":"2024-01-16T15:00:04.750064+0000","last_undegraded":"2024-01-16T15:00:04.750064+0000","last_fullsized":"2024-01-16T15:00:04.750064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:33:59.253631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.555160+0000","last_change":"2024-01-16T15:00:03.580412+0000","last_active":"2024-01-16T15:00:04.555160+0000","last_peered":"2024-01-16T15:00:04.555160+0000","last_clean":"2024-01-16T15:00:04.555160+0000","last_became_active":"2024-01-16T15:00:03.572852+0000","last_became_peered":"2024-01-16T15:00:03.572852+0000","last_unstale":"2024-01-16T15:00:04.555160+0000","last_undegraded":"2024-01-16T15:00:04.555160+0000","last_fullsized":"2024-01-16T15:00:04.555160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:48:44.736531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.613598+0000","last_change":"2024-01-16T15:00:02.635074+0000","last_active":"2024-01-16T15:00:03.613598+0000","last_peered":"2024-01-16T15:00:03.613598+0000","last_clean":"2024-01-16T15:00:03.613598+0000","last_became_active":"2024-01-16T15:00:02.634767+0000","last_became_peered":"2024-01-16T15:00:02.634767+0000","last_unstale":"2024-01-16T15:00:03.613598+0000","last_undegraded":"2024-01-16T15:00:03.613598+0000","last_fullsized":"2024-01-16T15:00:03.613598+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:15:58.991139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.324292+0000","last_change":"2024-01-16T15:00:03.573187+0000","last_active":"2024-01-16T15:00:05.324292+0000","last_peered":"2024-01-16T15:00:05.324292+0000","last_clean":"2024-01-16T15:00:05.324292+0000","last_became_active":"2024-01-16T15:00:03.572755+0000","last_became_peered":"2024-01-16T15:00:03.572755+0000","last_unstale":"2024-01-16T15:00:05.324292+0000","last_undegraded":"2024-01-16T15:00:05.324292+0000","last_fullsized":"2024-01-16T15:00:05.324292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:06:24.858403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.610883+0000","last_change":"2024-01-16T15:00:02.632028+0000","last_active":"2024-01-16T15:00:03.610883+0000","last_peered":"2024-01-16T15:00:03.610883+0000","last_clean":"2024-01-16T15:00:03.610883+0000","last_became_active":"2024-01-16T15:00:02.626019+0000","last_became_peered":"2024-01-16T15:00:02.626019+0000","last_unstale":"2024-01-16T15:00:03.610883+0000","last_undegraded":"2024-01-16T15:00:03.610883+0000","last_fullsized":"2024-01-16T15:00:03.610883+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:48:43.721848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.568172+0000","last_change":"2024-01-16T15:00:02.635363+0000","last_active":"2024-01-16T15:00:03.568172+0000","last_peered":"2024-01-16T15:00:03.568172+0000","last_clean":"2024-01-16T15:00:03.568172+0000","last_became_active":"2024-01-16T15:00:02.634387+0000","last_became_peered":"2024-01-16T15:00:02.634387+0000","last_unstale":"2024-01-16T15:00:03.568172+0000","last_undegraded":"2024-01-16T15:00:03.568172+0000","last_fullsized":"2024-01-16T15:00:03.568172+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:10:54.277062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.127245+0000","last_change":"2024-01-16T15:00:03.576575+0000","last_active":"2024-01-16T15:00:05.127245+0000","last_peered":"2024-01-16T15:00:05.127245+0000","last_clean":"2024-01-16T15:00:05.127245+0000","last_became_active":"2024-01-16T15:00:03.576266+0000","last_became_peered":"2024-01-16T15:00:03.576266+0000","last_unstale":"2024-01-16T15:00:05.127245+0000","last_undegraded":"2024-01-16T15:00:05.127245+0000","last_fullsized":"2024-01-16T15:00:05.127245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:28:03.212278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.584305+0000","last_change":"2024-01-16T15:00:02.636608+0000","last_active":"2024-01-16T15:00:03.584305+0000","last_peered":"2024-01-16T15:00:03.584305+0000","last_clean":"2024-01-16T15:00:03.584305+0000","last_became_active":"2024-01-16T15:00:02.636235+0000","last_became_peered":"2024-01-16T15:00:02.636235+0000","last_unstale":"2024-01-16T15:00:03.584305+0000","last_undegraded":"2024-01-16T15:00:03.584305+0000","last_fullsized":"2024-01-16T15:00:03.584305+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:11:10.313458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.553948+0000","last_change":"2024-01-16T15:00:03.579753+0000","last_active":"2024-01-16T15:00:04.553948+0000","last_peered":"2024-01-16T15:00:04.553948+0000","last_clean":"2024-01-16T15:00:04.553948+0000","last_became_active":"2024-01-16T15:00:03.579415+0000","last_became_peered":"2024-01-16T15:00:03.579415+0000","last_unstale":"2024-01-16T15:00:04.553948+0000","last_undegraded":"2024-01-16T15:00:04.553948+0000","last_fullsized":"2024-01-16T15:00:04.553948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:34:39.229613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.572357+0000","last_change":"2024-01-16T15:00:02.635549+0000","last_active":"2024-01-16T15:00:03.572357+0000","last_peered":"2024-01-16T15:00:03.572357+0000","last_clean":"2024-01-16T15:00:03.572357+0000","last_became_active":"2024-01-16T15:00:02.635309+0000","last_became_peered":"2024-01-16T15:00:02.635309+0000","last_unstale":"2024-01-16T15:00:03.572357+0000","last_undegraded":"2024-01-16T15:00:03.572357+0000","last_fullsized":"2024-01-16T15:00:03.572357+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:34:38.214930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323817+0000","last_change":"2024-01-16T15:00:03.586931+0000","last_active":"2024-01-16T15:00:05.323817+0000","last_peered":"2024-01-16T15:00:05.323817+0000","last_clean":"2024-01-16T15:00:05.323817+0000","last_became_active":"2024-01-16T15:00:03.586531+0000","last_became_peered":"2024-01-16T15:00:03.586531+0000","last_unstale":"2024-01-16T15:00:05.323817+0000","last_undegraded":"2024-01-16T15:00:05.323817+0000","last_fullsized":"2024-01-16T15:00:05.323817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:18:16.333187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.583406+0000","last_change":"2024-01-16T15:00:02.638637+0000","last_active":"2024-01-16T15:00:03.583406+0000","last_peered":"2024-01-16T15:00:03.583406+0000","last_clean":"2024-01-16T15:00:03.583406+0000","last_became_active":"2024-01-16T15:00:02.638355+0000","last_became_peered":"2024-01-16T15:00:02.638355+0000","last_unstale":"2024-01-16T15:00:03.583406+0000","last_undegraded":"2024-01-16T15:00:03.583406+0000","last_fullsized":"2024-01-16T15:00:03.583406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:53:48.731286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.324191+0000","last_change":"2024-01-16T15:00:03.587105+0000","last_active":"2024-01-16T15:00:05.324191+0000","last_peered":"2024-01-16T15:00:05.324191+0000","last_clean":"2024-01-16T15:00:05.324191+0000","last_became_active":"2024-01-16T15:00:03.586572+0000","last_became_peered":"2024-01-16T15:00:03.586572+0000","last_unstale":"2024-01-16T15:00:05.324191+0000","last_undegraded":"2024-01-16T15:00:05.324191+0000","last_fullsized":"2024-01-16T15:00:05.324191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:03:10.226190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.611665+0000","last_change":"2024-01-16T15:00:02.636352+0000","last_active":"2024-01-16T15:00:03.611665+0000","last_peered":"2024-01-16T15:00:03.611665+0000","last_clean":"2024-01-16T15:00:03.611665+0000","last_became_active":"2024-01-16T15:00:02.635069+0000","last_became_peered":"2024-01-16T15:00:02.635069+0000","last_unstale":"2024-01-16T15:00:03.611665+0000","last_undegraded":"2024-01-16T15:00:03.611665+0000","last_fullsized":"2024-01-16T15:00:03.611665+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:10:54.277062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.555540+0000","last_change":"2024-01-16T15:00:03.580722+0000","last_active":"2024-01-16T15:00:04.555540+0000","last_peered":"2024-01-16T15:00:04.555540+0000","last_clean":"2024-01-16T15:00:04.555540+0000","last_became_active":"2024-01-16T15:00:03.572307+0000","last_became_peered":"2024-01-16T15:00:03.572307+0000","last_unstale":"2024-01-16T15:00:04.555540+0000","last_undegraded":"2024-01-16T15:00:04.555540+0000","last_fullsized":"2024-01-16T15:00:04.555540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:25:28.863720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.617703+0000","last_change":"2024-01-16T15:00:02.625026+0000","last_active":"2024-01-16T15:00:03.617703+0000","last_peered":"2024-01-16T15:00:03.617703+0000","last_clean":"2024-01-16T15:00:03.617703+0000","last_became_active":"2024-01-16T15:00:02.622889+0000","last_became_peered":"2024-01-16T15:00:02.622889+0000","last_unstale":"2024-01-16T15:00:03.617703+0000","last_undegraded":"2024-01-16T15:00:03.617703+0000","last_fullsized":"2024-01-16T15:00:03.617703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:15:58.991139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.557063+0000","last_change":"2024-01-16T15:00:03.581190+0000","last_active":"2024-01-16T15:00:04.557063+0000","last_peered":"2024-01-16T15:00:04.557063+0000","last_clean":"2024-01-16T15:00:04.557063+0000","last_became_active":"2024-01-16T15:00:03.580903+0000","last_became_peered":"2024-01-16T15:00:03.580903+0000","last_unstale":"2024-01-16T15:00:04.557063+0000","last_undegraded":"2024-01-16T15:00:04.557063+0000","last_fullsized":"2024-01-16T15:00:04.557063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:17:11.329531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.615038+0000","last_change":"2024-01-16T15:00:02.622318+0000","last_active":"2024-01-16T15:00:03.615038+0000","last_peered":"2024-01-16T15:00:03.615038+0000","last_clean":"2024-01-16T15:00:03.615038+0000","last_became_active":"2024-01-16T15:00:02.622049+0000","last_became_peered":"2024-01-16T15:00:02.622049+0000","last_unstale":"2024-01-16T15:00:03.615038+0000","last_undegraded":"2024-01-16T15:00:03.615038+0000","last_fullsized":"2024-01-16T15:00:03.615038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:11:22.855828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323468+0000","last_change":"2024-01-16T15:00:03.574905+0000","last_active":"2024-01-16T15:00:05.323468+0000","last_peered":"2024-01-16T15:00:05.323468+0000","last_clean":"2024-01-16T15:00:05.323468+0000","last_became_active":"2024-01-16T15:00:03.572077+0000","last_became_peered":"2024-01-16T15:00:03.572077+0000","last_unstale":"2024-01-16T15:00:05.323468+0000","last_undegraded":"2024-01-16T15:00:05.323468+0000","last_fullsized":"2024-01-16T15:00:05.323468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:53:49.745969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.583051+0000","last_change":"2024-01-16T15:00:02.632349+0000","last_active":"2024-01-16T15:00:03.583051+0000","last_peered":"2024-01-16T15:00:03.583051+0000","last_clean":"2024-01-16T15:00:03.583051+0000","last_became_active":"2024-01-16T15:00:02.632038+0000","last_became_peered":"2024-01-16T15:00:02.632038+0000","last_unstale":"2024-01-16T15:00:03.583051+0000","last_undegraded":"2024-01-16T15:00:03.583051+0000","last_fullsized":"2024-01-16T15:00:03.583051+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:34:38.214930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.556735+0000","last_change":"2024-01-16T15:00:03.584616+0000","last_active":"2024-01-16T15:00:04.556735+0000","last_peered":"2024-01-16T15:00:04.556735+0000","last_clean":"2024-01-16T15:00:04.556735+0000","last_became_active":"2024-01-16T15:00:03.584348+0000","last_became_peered":"2024-01-16T15:00:03.584348+0000","last_unstale":"2024-01-16T15:00:04.556735+0000","last_undegraded":"2024-01-16T15:00:04.556735+0000","last_fullsized":"2024-01-16T15:00:04.556735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:18:16.333187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.569602+0000","last_change":"2024-01-16T15:00:02.628258+0000","last_active":"2024-01-16T15:00:03.569602+0000","last_peered":"2024-01-16T15:00:03.569602+0000","last_clean":"2024-01-16T15:00:03.569602+0000","last_became_active":"2024-01-16T15:00:02.627972+0000","last_became_peered":"2024-01-16T15:00:02.627972+0000","last_unstale":"2024-01-16T15:00:03.569602+0000","last_undegraded":"2024-01-16T15:00:03.569602+0000","last_fullsized":"2024-01-16T15:00:03.569602+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:25:27.849037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.750102+0000","last_change":"2024-01-16T15:00:03.588418+0000","last_active":"2024-01-16T15:00:04.750102+0000","last_peered":"2024-01-16T15:00:04.750102+0000","last_clean":"2024-01-16T15:00:04.750102+0000","last_became_active":"2024-01-16T15:00:03.587735+0000","last_became_peered":"2024-01-16T15:00:03.587735+0000","last_unstale":"2024-01-16T15:00:04.750102+0000","last_undegraded":"2024-01-16T15:00:04.750102+0000","last_fullsized":"2024-01-16T15:00:04.750102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:03:10.226190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.617567+0000","last_change":"2024-01-16T15:00:02.616381+0000","last_active":"2024-01-16T15:00:03.617567+0000","last_peered":"2024-01-16T15:00:03.617567+0000","last_clean":"2024-01-16T15:00:03.617567+0000","last_became_active":"2024-01-16T15:00:02.615244+0000","last_became_peered":"2024-01-16T15:00:02.615244+0000","last_unstale":"2024-01-16T15:00:03.617567+0000","last_undegraded":"2024-01-16T15:00:03.617567+0000","last_fullsized":"2024-01-16T15:00:03.617567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:21:47.228684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.556253+0000","last_change":"2024-01-16T15:00:03.584478+0000","last_active":"2024-01-16T15:00:04.556253+0000","last_peered":"2024-01-16T15:00:04.556253+0000","last_clean":"2024-01-16T15:00:04.556253+0000","last_became_active":"2024-01-16T15:00:03.583891+0000","last_became_peered":"2024-01-16T15:00:03.583891+0000","last_unstale":"2024-01-16T15:00:04.556253+0000","last_undegraded":"2024-01-16T15:00:04.556253+0000","last_fullsized":"2024-01-16T15:00:04.556253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:02:58.710795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.609993+0000","last_change":"2024-01-16T15:00:02.623236+0000","last_active":"2024-01-16T15:00:03.609993+0000","last_peered":"2024-01-16T15:00:03.609993+0000","last_clean":"2024-01-16T15:00:03.609993+0000","last_became_active":"2024-01-16T15:00:02.619961+0000","last_became_peered":"2024-01-16T15:00:02.619961+0000","last_unstale":"2024-01-16T15:00:03.609993+0000","last_undegraded":"2024-01-16T15:00:03.609993+0000","last_fullsized":"2024-01-16T15:00:03.609993+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:15:08.623284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.557756+0000","last_change":"2024-01-16T15:00:03.584135+0000","last_active":"2024-01-16T15:00:04.557756+0000","last_peered":"2024-01-16T15:00:04.557756+0000","last_clean":"2024-01-16T15:00:04.557756+0000","last_became_active":"2024-01-16T15:00:03.583742+0000","last_became_peered":"2024-01-16T15:00:03.583742+0000","last_unstale":"2024-01-16T15:00:04.557756+0000","last_undegraded":"2024-01-16T15:00:04.557756+0000","last_fullsized":"2024-01-16T15:00:04.557756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:54:11.712679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.613078+0000","last_change":"2024-01-16T15:00:02.625540+0000","last_active":"2024-01-16T15:00:03.613078+0000","last_peered":"2024-01-16T15:00:03.613078+0000","last_clean":"2024-01-16T15:00:03.613078+0000","last_became_active":"2024-01-16T15:00:02.625192+0000","last_became_peered":"2024-01-16T15:00:02.625192+0000","last_unstale":"2024-01-16T15:00:03.613078+0000","last_undegraded":"2024-01-16T15:00:03.613078+0000","last_fullsized":"2024-01-16T15:00:03.613078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:48:43.721848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.558230+0000","last_change":"2024-01-16T15:00:03.591505+0000","last_active":"2024-01-16T15:00:04.558230+0000","last_peered":"2024-01-16T15:00:04.558230+0000","last_clean":"2024-01-16T15:00:04.558230+0000","last_became_active":"2024-01-16T15:00:03.591187+0000","last_became_peered":"2024-01-16T15:00:03.591187+0000","last_unstale":"2024-01-16T15:00:04.558230+0000","last_undegraded":"2024-01-16T15:00:04.558230+0000","last_fullsized":"2024-01-16T15:00:04.558230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:05:33.003372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.610948+0000","last_change":"2024-01-16T15:00:02.627127+0000","last_active":"2024-01-16T15:00:03.610948+0000","last_peered":"2024-01-16T15:00:03.610948+0000","last_clean":"2024-01-16T15:00:03.610948+0000","last_became_active":"2024-01-16T15:00:02.624007+0000","last_became_peered":"2024-01-16T15:00:02.624007+0000","last_unstale":"2024-01-16T15:00:03.610948+0000","last_undegraded":"2024-01-16T15:00:03.610948+0000","last_fullsized":"2024-01-16T15:00:03.610948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:18:10.171130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323506+0000","last_change":"2024-01-16T15:00:03.588356+0000","last_active":"2024-01-16T15:00:05.323506+0000","last_peered":"2024-01-16T15:00:05.323506+0000","last_clean":"2024-01-16T15:00:05.323506+0000","last_became_active":"2024-01-16T15:00:03.588094+0000","last_became_peered":"2024-01-16T15:00:03.588094+0000","last_unstale":"2024-01-16T15:00:05.323506+0000","last_undegraded":"2024-01-16T15:00:05.323506+0000","last_fullsized":"2024-01-16T15:00:05.323506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:02:58.710795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.571707+0000","last_change":"2024-01-16T15:00:02.628534+0000","last_active":"2024-01-16T15:00:03.571707+0000","last_peered":"2024-01-16T15:00:03.571707+0000","last_clean":"2024-01-16T15:00:03.571707+0000","last_became_active":"2024-01-16T15:00:02.627252+0000","last_became_peered":"2024-01-16T15:00:02.627252+0000","last_unstale":"2024-01-16T15:00:03.571707+0000","last_undegraded":"2024-01-16T15:00:03.571707+0000","last_fullsized":"2024-01-16T15:00:03.571707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:17:47.960980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.556971+0000","last_change":"2024-01-16T15:00:03.588973+0000","last_active":"2024-01-16T15:00:04.556971+0000","last_peered":"2024-01-16T15:00:04.556971+0000","last_clean":"2024-01-16T15:00:04.556971+0000","last_became_active":"2024-01-16T15:00:03.588673+0000","last_became_peered":"2024-01-16T15:00:03.588673+0000","last_unstale":"2024-01-16T15:00:04.556971+0000","last_undegraded":"2024-01-16T15:00:04.556971+0000","last_fullsized":"2024-01-16T15:00:04.556971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:29:26.138536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.554975+0000","last_change":"2024-01-16T14:59:53.430715+0000","last_active":"2024-01-16T15:00:03.554975+0000","last_peered":"2024-01-16T15:00:03.554975+0000","last_clean":"2024-01-16T15:00:03.554975+0000","last_became_active":"2024-01-16T14:59:53.428209+0000","last_became_peered":"2024-01-16T14:59:53.428209+0000","last_unstale":"2024-01-16T15:00:03.554975+0000","last_undegraded":"2024-01-16T15:00:03.554975+0000","last_fullsized":"2024-01-16T15:00:03.554975+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:59:52.011232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:59:52.011232+0000","last_clean_scrub_stamp":"2024-01-16T14:59:52.011232+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:57:42.594848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.585945+0000","last_change":"2024-01-16T15:00:02.635313+0000","last_active":"2024-01-16T15:00:03.585945+0000","last_peered":"2024-01-16T15:00:03.585945+0000","last_clean":"2024-01-16T15:00:03.585945+0000","last_became_active":"2024-01-16T15:00:02.634804+0000","last_became_peered":"2024-01-16T15:00:02.634804+0000","last_unstale":"2024-01-16T15:00:03.585945+0000","last_undegraded":"2024-01-16T15:00:03.585945+0000","last_fullsized":"2024-01-16T15:00:03.585945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:28:02.197595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.127292+0000","last_change":"2024-01-16T15:00:03.590575+0000","last_active":"2024-01-16T15:00:05.127292+0000","last_peered":"2024-01-16T15:00:05.127292+0000","last_clean":"2024-01-16T15:00:05.127292+0000","last_became_active":"2024-01-16T15:00:03.590218+0000","last_became_peered":"2024-01-16T15:00:03.590218+0000","last_unstale":"2024-01-16T15:00:05.127292+0000","last_undegraded":"2024-01-16T15:00:05.127292+0000","last_fullsized":"2024-01-16T15:00:05.127292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:33:59.253631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.555195+0000","last_change":"2024-01-16T15:00:02.636675+0000","last_active":"2024-01-16T15:00:03.555195+0000","last_peered":"2024-01-16T15:00:03.555195+0000","last_clean":"2024-01-16T15:00:03.555195+0000","last_became_active":"2024-01-16T15:00:02.633102+0000","last_became_peered":"2024-01-16T15:00:02.633102+0000","last_unstale":"2024-01-16T15:00:03.555195+0000","last_undegraded":"2024-01-16T15:00:03.555195+0000","last_fullsized":"2024-01-16T15:00:03.555195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:57:13.913346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323042+0000","last_change":"2024-01-16T15:00:03.583866+0000","last_active":"2024-01-16T15:00:05.323042+0000","last_peered":"2024-01-16T15:00:05.323042+0000","last_clean":"2024-01-16T15:00:05.323042+0000","last_became_active":"2024-01-16T15:00:03.583561+0000","last_became_peered":"2024-01-16T15:00:03.583561+0000","last_unstale":"2024-01-16T15:00:05.323042+0000","last_undegraded":"2024-01-16T15:00:05.323042+0000","last_fullsized":"2024-01-16T15:00:05.323042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:25:28.863720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.571916+0000","last_change":"2024-01-16T15:00:02.624133+0000","last_active":"2024-01-16T15:00:03.571916+0000","last_peered":"2024-01-16T15:00:03.571916+0000","last_clean":"2024-01-16T15:00:03.571916+0000","last_became_active":"2024-01-16T15:00:02.623883+0000","last_became_peered":"2024-01-16T15:00:02.623883+0000","last_unstale":"2024-01-16T15:00:03.571916+0000","last_undegraded":"2024-01-16T15:00:03.571916+0000","last_fullsized":"2024-01-16T15:00:03.571916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:00:52.721183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.554313+0000","last_change":"2024-01-16T15:00:03.571151+0000","last_active":"2024-01-16T15:00:04.554313+0000","last_peered":"2024-01-16T15:00:04.554313+0000","last_clean":"2024-01-16T15:00:04.554313+0000","last_became_active":"2024-01-16T15:00:03.570800+0000","last_became_peered":"2024-01-16T15:00:03.570800+0000","last_unstale":"2024-01-16T15:00:04.554313+0000","last_undegraded":"2024-01-16T15:00:04.554313+0000","last_fullsized":"2024-01-16T15:00:04.554313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:03:10.226190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.586154+0000","last_change":"2024-01-16T15:00:02.637246+0000","last_active":"2024-01-16T15:00:03.586154+0000","last_peered":"2024-01-16T15:00:03.586154+0000","last_clean":"2024-01-16T15:00:03.586154+0000","last_became_active":"2024-01-16T15:00:02.636465+0000","last_became_peered":"2024-01-16T15:00:02.636465+0000","last_unstale":"2024-01-16T15:00:03.586154+0000","last_undegraded":"2024-01-16T15:00:03.586154+0000","last_fullsized":"2024-01-16T15:00:03.586154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:33:58.238948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.555119+0000","last_change":"2024-01-16T15:00:03.580644+0000","last_active":"2024-01-16T15:00:04.555119+0000","last_peered":"2024-01-16T15:00:04.555119+0000","last_clean":"2024-01-16T15:00:04.555119+0000","last_became_active":"2024-01-16T15:00:03.573120+0000","last_became_peered":"2024-01-16T15:00:03.573120+0000","last_unstale":"2024-01-16T15:00:04.555119+0000","last_undegraded":"2024-01-16T15:00:04.555119+0000","last_fullsized":"2024-01-16T15:00:04.555119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:21:48.243367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.563287+0000","last_change":"2024-01-16T15:00:02.620390+0000","last_active":"2024-01-16T15:00:03.563287+0000","last_peered":"2024-01-16T15:00:03.563287+0000","last_clean":"2024-01-16T15:00:03.563287+0000","last_became_active":"2024-01-16T15:00:02.619046+0000","last_became_peered":"2024-01-16T15:00:02.619046+0000","last_unstale":"2024-01-16T15:00:03.563287+0000","last_undegraded":"2024-01-16T15:00:03.563287+0000","last_fullsized":"2024-01-16T15:00:03.563287+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:48:43.721848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323450+0000","last_change":"2024-01-16T15:00:03.587239+0000","last_active":"2024-01-16T15:00:05.323450+0000","last_peered":"2024-01-16T15:00:05.323450+0000","last_clean":"2024-01-16T15:00:05.323450+0000","last_became_active":"2024-01-16T15:00:03.586925+0000","last_became_peered":"2024-01-16T15:00:03.586925+0000","last_unstale":"2024-01-16T15:00:05.323450+0000","last_undegraded":"2024-01-16T15:00:05.323450+0000","last_fullsized":"2024-01-16T15:00:05.323450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:09:47.428018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.555099+0000","last_change":"2024-01-16T15:00:02.640889+0000","last_active":"2024-01-16T15:00:03.555099+0000","last_peered":"2024-01-16T15:00:03.555099+0000","last_clean":"2024-01-16T15:00:03.555099+0000","last_became_active":"2024-01-16T15:00:02.633676+0000","last_became_peered":"2024-01-16T15:00:02.633676+0000","last_unstale":"2024-01-16T15:00:03.555099+0000","last_undegraded":"2024-01-16T15:00:03.555099+0000","last_fullsized":"2024-01-16T15:00:03.555099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:03:09.211508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.554126+0000","last_change":"2024-01-16T15:00:03.576742+0000","last_active":"2024-01-16T15:00:04.554126+0000","last_peered":"2024-01-16T15:00:04.554126+0000","last_clean":"2024-01-16T15:00:04.554126+0000","last_became_active":"2024-01-16T15:00:03.576104+0000","last_became_peered":"2024-01-16T15:00:03.576104+0000","last_unstale":"2024-01-16T15:00:04.554126+0000","last_undegraded":"2024-01-16T15:00:04.554126+0000","last_fullsized":"2024-01-16T15:00:04.554126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:28:03.212278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.564812+0000","last_change":"2024-01-16T15:00:02.633317+0000","last_active":"2024-01-16T15:00:03.564812+0000","last_peered":"2024-01-16T15:00:03.564812+0000","last_clean":"2024-01-16T15:00:03.564812+0000","last_became_active":"2024-01-16T15:00:02.632996+0000","last_became_peered":"2024-01-16T15:00:02.632996+0000","last_unstale":"2024-01-16T15:00:03.564812+0000","last_undegraded":"2024-01-16T15:00:03.564812+0000","last_fullsized":"2024-01-16T15:00:03.564812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:53:48.731286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.128153+0000","last_change":"2024-01-16T15:00:03.577601+0000","last_active":"2024-01-16T15:00:05.128153+0000","last_peered":"2024-01-16T15:00:05.128153+0000","last_clean":"2024-01-16T15:00:05.128153+0000","last_became_active":"2024-01-16T15:00:03.577107+0000","last_became_peered":"2024-01-16T15:00:03.577107+0000","last_unstale":"2024-01-16T15:00:05.128153+0000","last_undegraded":"2024-01-16T15:00:05.128153+0000","last_fullsized":"2024-01-16T15:00:05.128153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:31:09.422997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.612223+0000","last_change":"2024-01-16T15:00:02.634633+0000","last_active":"2024-01-16T15:00:03.612223+0000","last_peered":"2024-01-16T15:00:03.612223+0000","last_clean":"2024-01-16T15:00:03.612223+0000","last_became_active":"2024-01-16T15:00:02.634372+0000","last_became_peered":"2024-01-16T15:00:02.634372+0000","last_unstale":"2024-01-16T15:00:03.612223+0000","last_undegraded":"2024-01-16T15:00:03.612223+0000","last_fullsized":"2024-01-16T15:00:03.612223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:11:10.313458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.324092+0000","last_change":"2024-01-16T15:00:03.575017+0000","last_active":"2024-01-16T15:00:05.324092+0000","last_peered":"2024-01-16T15:00:05.324092+0000","last_clean":"2024-01-16T15:00:05.324092+0000","last_became_active":"2024-01-16T15:00:03.573750+0000","last_became_peered":"2024-01-16T15:00:03.573750+0000","last_unstale":"2024-01-16T15:00:05.324092+0000","last_undegraded":"2024-01-16T15:00:05.324092+0000","last_fullsized":"2024-01-16T15:00:05.324092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:28:03.212278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.552344+0000","last_change":"2024-01-16T15:00:02.636963+0000","last_active":"2024-01-16T15:00:03.552344+0000","last_peered":"2024-01-16T15:00:03.552344+0000","last_clean":"2024-01-16T15:00:03.552344+0000","last_became_active":"2024-01-16T15:00:02.632840+0000","last_became_peered":"2024-01-16T15:00:02.632840+0000","last_unstale":"2024-01-16T15:00:03.552344+0000","last_undegraded":"2024-01-16T15:00:03.552344+0000","last_fullsized":"2024-01-16T15:00:03.552344+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:21:47.228684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.554498+0000","last_change":"2024-01-16T15:00:03.582851+0000","last_active":"2024-01-16T15:00:04.554498+0000","last_peered":"2024-01-16T15:00:04.554498+0000","last_clean":"2024-01-16T15:00:04.554498+0000","last_became_active":"2024-01-16T15:00:03.581868+0000","last_became_peered":"2024-01-16T15:00:03.581868+0000","last_unstale":"2024-01-16T15:00:04.554498+0000","last_undegraded":"2024-01-16T15:00:04.554498+0000","last_fullsized":"2024-01-16T15:00:04.554498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:00:53.735866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.563577+0000","last_change":"2024-01-16T15:00:02.622029+0000","last_active":"2024-01-16T15:00:03.563577+0000","last_peered":"2024-01-16T15:00:03.563577+0000","last_clean":"2024-01-16T15:00:03.563577+0000","last_became_active":"2024-01-16T15:00:02.620427+0000","last_became_peered":"2024-01-16T15:00:02.620427+0000","last_unstale":"2024-01-16T15:00:03.563577+0000","last_undegraded":"2024-01-16T15:00:03.563577+0000","last_fullsized":"2024-01-16T15:00:03.563577+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:25:27.849037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.127523+0000","last_change":"2024-01-16T15:00:03.580221+0000","last_active":"2024-01-16T15:00:05.127523+0000","last_peered":"2024-01-16T15:00:05.127523+0000","last_clean":"2024-01-16T15:00:05.127523+0000","last_became_active":"2024-01-16T15:00:03.579960+0000","last_became_peered":"2024-01-16T15:00:03.579960+0000","last_unstale":"2024-01-16T15:00:05.127523+0000","last_undegraded":"2024-01-16T15:00:05.127523+0000","last_fullsized":"2024-01-16T15:00:05.127523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:03:10.226190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.564091+0000","last_change":"2024-01-16T15:00:02.631776+0000","last_active":"2024-01-16T15:00:03.564091+0000","last_peered":"2024-01-16T15:00:03.564091+0000","last_clean":"2024-01-16T15:00:03.564091+0000","last_became_active":"2024-01-16T15:00:02.630658+0000","last_became_peered":"2024-01-16T15:00:02.630658+0000","last_unstale":"2024-01-16T15:00:03.564091+0000","last_undegraded":"2024-01-16T15:00:03.564091+0000","last_fullsized":"2024-01-16T15:00:03.564091+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:15:58.991139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.128076+0000","last_change":"2024-01-16T15:00:03.574061+0000","last_active":"2024-01-16T15:00:05.128076+0000","last_peered":"2024-01-16T15:00:05.128076+0000","last_clean":"2024-01-16T15:00:05.128076+0000","last_became_active":"2024-01-16T15:00:03.573691+0000","last_became_peered":"2024-01-16T15:00:03.573691+0000","last_unstale":"2024-01-16T15:00:05.128076+0000","last_undegraded":"2024-01-16T15:00:05.128076+0000","last_fullsized":"2024-01-16T15:00:05.128076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:05:08.127179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.615335+0000","last_change":"2024-01-16T15:00:02.634122+0000","last_active":"2024-01-16T15:00:03.615335+0000","last_peered":"2024-01-16T15:00:03.615335+0000","last_clean":"2024-01-16T15:00:03.615335+0000","last_became_active":"2024-01-16T15:00:02.633819+0000","last_became_peered":"2024-01-16T15:00:02.633819+0000","last_unstale":"2024-01-16T15:00:03.615335+0000","last_undegraded":"2024-01-16T15:00:03.615335+0000","last_fullsized":"2024-01-16T15:00:03.615335+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:53:48.731286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.750129+0000","last_change":"2024-01-16T15:00:03.590510+0000","last_active":"2024-01-16T15:00:04.750129+0000","last_peered":"2024-01-16T15:00:04.750129+0000","last_clean":"2024-01-16T15:00:04.750129+0000","last_became_active":"2024-01-16T15:00:03.588555+0000","last_became_peered":"2024-01-16T15:00:03.588555+0000","last_unstale":"2024-01-16T15:00:04.750129+0000","last_undegraded":"2024-01-16T15:00:04.750129+0000","last_fullsized":"2024-01-16T15:00:04.750129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:06:24.858403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323387+0000","last_change":"2024-01-16T15:00:03.594894+0000","last_active":"2024-01-16T15:00:05.323387+0000","last_peered":"2024-01-16T15:00:05.323387+0000","last_clean":"2024-01-16T15:00:05.323387+0000","last_became_active":"2024-01-16T15:00:03.592616+0000","last_became_peered":"2024-01-16T15:00:03.592616+0000","last_unstale":"2024-01-16T15:00:05.323387+0000","last_undegraded":"2024-01-16T15:00:05.323387+0000","last_fullsized":"2024-01-16T15:00:05.323387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:31:09.422997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.615281+0000","last_change":"2024-01-16T15:00:02.631757+0000","last_active":"2024-01-16T15:00:03.615281+0000","last_peered":"2024-01-16T15:00:03.615281+0000","last_clean":"2024-01-16T15:00:03.615281+0000","last_became_active":"2024-01-16T15:00:02.630946+0000","last_became_peered":"2024-01-16T15:00:02.630946+0000","last_unstale":"2024-01-16T15:00:03.615281+0000","last_undegraded":"2024-01-16T15:00:03.615281+0000","last_fullsized":"2024-01-16T15:00:03.615281+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:34:38.214930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323082+0000","last_change":"2024-01-16T15:00:03.593210+0000","last_active":"2024-01-16T15:00:05.323082+0000","last_peered":"2024-01-16T15:00:05.323082+0000","last_clean":"2024-01-16T15:00:05.323082+0000","last_became_active":"2024-01-16T15:00:03.592630+0000","last_became_peered":"2024-01-16T15:00:03.592630+0000","last_unstale":"2024-01-16T15:00:05.323082+0000","last_undegraded":"2024-01-16T15:00:05.323082+0000","last_fullsized":"2024-01-16T15:00:05.323082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:05:08.127179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.615363+0000","last_change":"2024-01-16T15:00:02.641351+0000","last_active":"2024-01-16T15:00:03.615363+0000","last_peered":"2024-01-16T15:00:03.615363+0000","last_clean":"2024-01-16T15:00:03.615363+0000","last_became_active":"2024-01-16T15:00:02.641021+0000","last_became_peered":"2024-01-16T15:00:02.641021+0000","last_unstale":"2024-01-16T15:00:03.615363+0000","last_undegraded":"2024-01-16T15:00:03.615363+0000","last_fullsized":"2024-01-16T15:00:03.615363+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:11:10.313458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.324164+0000","last_change":"2024-01-16T15:00:03.596490+0000","last_active":"2024-01-16T15:00:05.324164+0000","last_peered":"2024-01-16T15:00:05.324164+0000","last_clean":"2024-01-16T15:00:05.324164+0000","last_became_active":"2024-01-16T15:00:03.596137+0000","last_became_peered":"2024-01-16T15:00:03.596137+0000","last_unstale":"2024-01-16T15:00:05.324164+0000","last_undegraded":"2024-01-16T15:00:05.324164+0000","last_fullsized":"2024-01-16T15:00:05.324164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:33:59.253631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.612476+0000","last_change":"2024-01-16T15:00:02.630103+0000","last_active":"2024-01-16T15:00:03.612476+0000","last_peered":"2024-01-16T15:00:03.612476+0000","last_clean":"2024-01-16T15:00:03.612476+0000","last_became_active":"2024-01-16T15:00:02.629827+0000","last_became_peered":"2024-01-16T15:00:02.629827+0000","last_unstale":"2024-01-16T15:00:03.612476+0000","last_undegraded":"2024-01-16T15:00:03.612476+0000","last_fullsized":"2024-01-16T15:00:03.612476+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:25:27.849037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.127755+0000","last_change":"2024-01-16T15:00:03.583878+0000","last_active":"2024-01-16T15:00:05.127755+0000","last_peered":"2024-01-16T15:00:05.127755+0000","last_clean":"2024-01-16T15:00:05.127755+0000","last_became_active":"2024-01-16T15:00:03.583033+0000","last_became_peered":"2024-01-16T15:00:03.583033+0000","last_unstale":"2024-01-16T15:00:05.127755+0000","last_undegraded":"2024-01-16T15:00:05.127755+0000","last_fullsized":"2024-01-16T15:00:05.127755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:57:14.928029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.616725+0000","last_change":"2024-01-16T15:00:02.634166+0000","last_active":"2024-01-16T15:00:03.616725+0000","last_peered":"2024-01-16T15:00:03.616725+0000","last_clean":"2024-01-16T15:00:03.616725+0000","last_became_active":"2024-01-16T15:00:02.633874+0000","last_became_peered":"2024-01-16T15:00:02.633874+0000","last_unstale":"2024-01-16T15:00:03.616725+0000","last_undegraded":"2024-01-16T15:00:03.616725+0000","last_fullsized":"2024-01-16T15:00:03.616725+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:39:13.413754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1719999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2809999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.419}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.181}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5649999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7350000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6779999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4450000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9750000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1619999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2190000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6180000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7230000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.897}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.99}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.093}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.149}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2090000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.528}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7470000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0289999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.968}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0670000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0449999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.179}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3380000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.873}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5489999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1419999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1719999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95299999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1910000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":806912,"data_stored":660897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.135}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2949999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2709999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.323}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-16T15:00:07.984 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-16T15:00:08.094 INFO:tasks.ceph.mds.a.smithi143.stderr:2024-01-16T15:00:08.114+0000 7fae90200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-16T15:00:08.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.245+0000 7fb6fdc00640 1 Processor -- start 2024-01-16T15:00:08.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.245+0000 7fb6fdc00640 1 -- start start 2024-01-16T15:00:08.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.247+0000 7fb6fdc00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb6f80af490 0x7fb6f806ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:08.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.247+0000 7fb6fdc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb6f80cbc70 0x7fb6f815f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:08.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.247+0000 7fb6fdc00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6f80cd9c0 con 0x7fb6f80af870 2024-01-16T15:00:08.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.247+0000 7fb6fdc00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6f80cce90 con 0x7fb6f80af490 2024-01-16T15:00:08.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.247+0000 7fb6fdc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f80cc210 con 0x7fb6f80cbc70 2024-01-16T15:00:08.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.247+0000 7fb6f6000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb6f80af870 0x7fb6f80cd2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54082/0 (socket says 172.21.15.143:54082) 2024-01-16T15:00:08.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.247+0000 7fb6f6000640 1 -- 172.21.15.143:0/523757287 learned_addr learned my addr 172.21.15.143:0/523757287 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:08.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.247+0000 7fb6f6a00640 1 --2- 172.21.15.143:0/523757287 >> v2:172.21.15.204:3300/0 conn(0x7fb6f80af490 0x7fb6f806ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:08.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.248+0000 7fb6f7400640 1 --2- 172.21.15.143:0/523757287 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb6f80cbc70 0x7fb6f815f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:08.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.248+0000 7fb6f5600640 1 -- 172.21.15.143:0/523757287 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4058800793 0 0) 0x7fb6f80cd9c0 con 0x7fb6f80af870 2024-01-16T15:00:08.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.248+0000 7fb6f5600640 1 -- 172.21.15.143:0/523757287 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6d8003520 con 0x7fb6f80af870 2024-01-16T15:00:08.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.248+0000 7fb6f5600640 1 -- 172.21.15.143:0/523757287 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3313860285 0 0) 0x7fb6d8003520 con 0x7fb6f80af870 2024-01-16T15:00:08.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.248+0000 7fb6f5600640 1 -- 172.21.15.143:0/523757287 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb6f80cbc70 msgr2=0x7fb6f815f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:08.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.248+0000 7fb6f5600640 1 --2- 172.21.15.143:0/523757287 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb6f80cbc70 0x7fb6f815f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:08.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.248+0000 7fb6f5600640 1 -- 172.21.15.143:0/523757287 >> v2:172.21.15.204:3300/0 conn(0x7fb6f80af490 msgr2=0x7fb6f806ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:08.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.248+0000 7fb6f5600640 1 --2- 172.21.15.143:0/523757287 >> v2:172.21.15.204:3300/0 conn(0x7fb6f80af490 0x7fb6f806ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:08.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.248+0000 7fb6f5600640 1 -- 172.21.15.143:0/523757287 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f80cc4f0 con 0x7fb6f80af870 2024-01-16T15:00:08.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.248+0000 7fb6f5600640 1 -- 172.21.15.143:0/523757287 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7fb6e00019f0 con 0x7fb6f80af870 2024-01-16T15:00:08.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.248+0000 7fb6f5600640 1 -- 172.21.15.143:0/523757287 >> v1:172.21.15.143:6789/0 conn(0x7fb6f80af870 legacy=0x7fb6f80cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:08.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.248+0000 7fb6f5600640 1 --2- 172.21.15.143:0/523757287 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb6f80cbc70 0x7fb6d8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:08.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.248+0000 7fb6f5600640 1 --2- 172.21.15.143:0/523757287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6d8003f40 0x7fb6d80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:08.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.248+0000 7fb6f5600640 1 --2- 172.21.15.143:0/523757287 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6d8004d00 0x7fb6d8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:08.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.248+0000 7fb6f5600640 1 -- 172.21.15.143:0/523757287 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f80cce90 con 0x7fb6d8004d00 2024-01-16T15:00:08.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.248+0000 7fb6f5600640 1 -- 172.21.15.143:0/523757287 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f80cc210 con 0x7fb6d8003f40 2024-01-16T15:00:08.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.248+0000 7fb6f5600640 1 -- 172.21.15.143:0/523757287 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d800b980 con 0x7fb6f80cbc70 2024-01-16T15:00:08.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.248+0000 7fb6f7400640 1 --2- 172.21.15.143:0/523757287 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6d8004d00 0x7fb6d8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:08.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.249+0000 7fb6f6000640 1 --2- 172.21.15.143:0/523757287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6d8003f40 0x7fb6d80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:08.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.249+0000 7fb6f7400640 1 -- 172.21.15.143:0/523757287 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb6f80cbc70 msgr2=0x7fb6d8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:08.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.249+0000 7fb6f6a00640 1 --2- 172.21.15.143:0/523757287 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb6f80cbc70 0x7fb6d8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:08.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.249+0000 7fb6f7400640 1 --2- 172.21.15.143:0/523757287 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb6f80cbc70 0x7fb6d8003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:08.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.249+0000 7fb6f7400640 1 -- 172.21.15.143:0/523757287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6d8003f40 msgr2=0x7fb6d80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:08.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.249+0000 7fb6f7400640 1 --2- 172.21.15.143:0/523757287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6d8003f40 0x7fb6d80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:08.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.249+0000 7fb6f7400640 1 -- 172.21.15.143:0/523757287 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d8003520 con 0x7fb6d8004d00 2024-01-16T15:00:08.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.249+0000 7fb6f6000640 1 --2- 172.21.15.143:0/523757287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6d8003f40 0x7fb6d80047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:08.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.250+0000 7fb6f7400640 1 --2- 172.21.15.143:0/523757287 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6d8004d00 0x7fb6d8005180 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0009af0 tx=0x7fb6d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d38517f844730 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:08.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.250+0000 7fb6f5600640 1 -- 172.21.15.143:0/523757287 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f002c6b0 con 0x7fb6d8004d00 2024-01-16T15:00:08.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.250+0000 7fb6f5600640 1 -- 172.21.15.143:0/523757287 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f002c810 con 0x7fb6d8004d00 2024-01-16T15:00:08.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.250+0000 7fb6f5600640 1 -- 172.21.15.143:0/523757287 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f002cba0 con 0x7fb6d8004d00 2024-01-16T15:00:08.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.251+0000 7fb6fdc00640 1 -- 172.21.15.143:0/523757287 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6d8004d00 msgr2=0x7fb6d8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:08.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.251+0000 7fb6fdc00640 1 --2- 172.21.15.143:0/523757287 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6d8004d00 0x7fb6d8005180 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0009af0 tx=0x7fb6d80057f0 comp rx=0 tx=0).stop 2024-01-16T15:00:08.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.251+0000 7fb6f7400640 1 -- 172.21.15.143:0/523757287 reap_dead start 2024-01-16T15:00:08.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.251+0000 7fb6fdc00640 1 -- 172.21.15.143:0/523757287 shutdown_connections 2024-01-16T15:00:08.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.251+0000 7fb6fdc00640 1 -- 172.21.15.143:0/523757287 >> 172.21.15.143:0/523757287 conn(0x7fb6f80999d0 msgr2=0x7fb6f80b1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:08.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.251+0000 7fb6fdc00640 1 -- 172.21.15.143:0/523757287 shutdown_connections 2024-01-16T15:00:08.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.251+0000 7fb6fdc00640 1 -- 172.21.15.143:0/523757287 wait complete. 2024-01-16T15:00:08.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.252+0000 7fb6fdc00640 1 Processor -- start 2024-01-16T15:00:08.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.253+0000 7fb6fdc00640 1 -- start start 2024-01-16T15:00:08.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.253+0000 7fb6fdc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb6d8004d00 0x7fb6f80ae290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:08.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.253+0000 7fb6fdc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6f814f080 0x7fb6f814f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:08.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.253+0000 7fb6fdc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6f80c91d0 0x7fb6f80c9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:08.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.253+0000 7fb6fdc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f81f67c0 con 0x7fb6f814f080 2024-01-16T15:00:08.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.253+0000 7fb6fdc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f81f6aa0 con 0x7fb6f80c91d0 2024-01-16T15:00:08.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.253+0000 7fb6fdc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f81f6d80 con 0x7fb6d8004d00 2024-01-16T15:00:08.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.254+0000 7fb6f6000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6f814f080 0x7fb6f814f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:08.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.254+0000 7fb6f6000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6f814f080 0x7fb6f814f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42424/0 (socket says 172.21.15.143:42424) 2024-01-16T15:00:08.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.254+0000 7fb6f6000640 1 -- 172.21.15.143:0/133143044 learned_addr learned my addr 172.21.15.143:0/133143044 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:08.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.254+0000 7fb6f6a00640 1 --2- 172.21.15.143:0/133143044 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb6d8004d00 0x7fb6f80ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:08.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.254+0000 7fb6f6000640 1 -- 172.21.15.143:0/133143044 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb6d8004d00 msgr2=0x7fb6f80ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:08.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.254+0000 7fb6f6000640 1 --2- 172.21.15.143:0/133143044 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb6d8004d00 0x7fb6f80ae290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:08.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.254+0000 7fb6f6000640 1 -- 172.21.15.143:0/133143044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6f80c91d0 msgr2=0x7fb6f80c9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:08.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.254+0000 7fb6f6000640 1 --2- 172.21.15.143:0/133143044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6f80c91d0 0x7fb6f80c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:08.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.254+0000 7fb6f6000640 1 -- 172.21.15.143:0/133143044 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f81f7060 con 0x7fb6f814f080 2024-01-16T15:00:08.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.254+0000 7fb6f6000640 1 --2- 172.21.15.143:0/133143044 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6f814f080 0x7fb6f814f4e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb6e000b0d0 tx=0x7fb6e000b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:08.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.254+0000 7fb6eea00640 1 -- 172.21.15.143:0/133143044 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0004d60 con 0x7fb6f814f080 2024-01-16T15:00:08.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.255+0000 7fb6eea00640 1 -- 172.21.15.143:0/133143044 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6e00070d0 con 0x7fb6f814f080 2024-01-16T15:00:08.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.255+0000 7fb6eea00640 1 -- 172.21.15.143:0/133143044 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0007440 con 0x7fb6f814f080 2024-01-16T15:00:08.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.255+0000 7fb6fdc00640 1 -- 172.21.15.143:0/133143044 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6f814fa20 con 0x7fb6f814f080 2024-01-16T15:00:08.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.255+0000 7fb6fdc00640 1 -- 172.21.15.143:0/133143044 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f80ae9b0 con 0x7fb6f814f080 2024-01-16T15:00:08.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.256+0000 7fb6fdc00640 1 -- 172.21.15.143:0/133143044 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6f815fdd0 con 0x7fb6f814f080 2024-01-16T15:00:08.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.257+0000 7fb6eea00640 1 -- 172.21.15.143:0/133143044 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fb6e00075a0 con 0x7fb6f814f080 2024-01-16T15:00:08.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.257+0000 7fb6eea00640 1 --2- 172.21.15.143:0/133143044 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb6d006b2c0 0x7fb6d006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:08.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.258+0000 7fb6eea00640 1 -- 172.21.15.143:0/133143044 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb6e0089ae0 con 0x7fb6f814f080 2024-01-16T15:00:08.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.258+0000 7fb6f6a00640 1 --2- 172.21.15.143:0/133143044 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb6d006b2c0 0x7fb6d006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:08.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.258+0000 7fb6f6a00640 1 --2- 172.21.15.143:0/133143044 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb6d006b2c0 0x7fb6d006d780 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009990 tx=0x7fb6e8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:08.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.262+0000 7fb6eea00640 1 -- 172.21.15.143:0/133143044 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb6e0054940 con 0x7fb6f814f080 2024-01-16T15:00:08.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.434+0000 7fb6fdc00640 1 -- 172.21.15.143:0/133143044 --> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb6f8155270 con 0x7fb6d006b2c0 2024-01-16T15:00:08.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.442+0000 7fb6eea00640 1 -- 172.21.15.143:0/133143044 <== mgr.4113 v2:172.21.15.143:6832/3153492115 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264522 (secure 0 0 0) 0x7fb6f8155270 con 0x7fb6d006b2c0 2024-01-16T15:00:08.426 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:00:08.427 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-16T15:00:08.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.447+0000 7fb6fdc00640 1 -- 172.21.15.143:0/133143044 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb6d006b2c0 msgr2=0x7fb6d006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:08.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.447+0000 7fb6fdc00640 1 --2- 172.21.15.143:0/133143044 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb6d006b2c0 0x7fb6d006d780 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009990 tx=0x7fb6e8008040 comp rx=0 tx=0).stop 2024-01-16T15:00:08.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.447+0000 7fb6fdc00640 1 -- 172.21.15.143:0/133143044 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6f814f080 msgr2=0x7fb6f814f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:08.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.447+0000 7fb6fdc00640 1 --2- 172.21.15.143:0/133143044 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6f814f080 0x7fb6f814f4e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb6e000b0d0 tx=0x7fb6e000b590 comp rx=0 tx=0).stop 2024-01-16T15:00:08.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.447+0000 7fb6fdc00640 1 -- 172.21.15.143:0/133143044 shutdown_connections 2024-01-16T15:00:08.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.447+0000 7fb6fdc00640 1 --2- 172.21.15.143:0/133143044 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb6d006b2c0 0x7fb6d006d780 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:08.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.447+0000 7fb6fdc00640 1 --2- 172.21.15.143:0/133143044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6f80c91d0 0x7fb6f80c9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:08.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.447+0000 7fb6fdc00640 1 --2- 172.21.15.143:0/133143044 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6f814f080 0x7fb6f814f4e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:08.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.447+0000 7fb6fdc00640 1 --2- 172.21.15.143:0/133143044 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb6d8004d00 0x7fb6f80ae290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:08.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.447+0000 7fb6fdc00640 1 -- 172.21.15.143:0/133143044 >> 172.21.15.143:0/133143044 conn(0x7fb6f80999d0 msgr2=0x7fb6f80d00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:08.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.447+0000 7fb6fdc00640 1 -- 172.21.15.143:0/133143044 shutdown_connections 2024-01-16T15:00:08.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.447+0000 7fb6fdc00640 1 -- 172.21.15.143:0/133143044 wait complete. 2024-01-16T15:00:08.444 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-01-16T15:00:07.653854+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":188,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219004,"kb_used_data":2820,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755716,"statfs":{"total":773094113280,"available":772869853184,"internally_reserved":0,"allocated":2887680,"data_stored":1751817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":58,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001036"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.558033+0000","last_change":"2024-01-16T15:00:03.579227+0000","last_active":"2024-01-16T15:00:04.558033+0000","last_peered":"2024-01-16T15:00:04.558033+0000","last_clean":"2024-01-16T15:00:04.558033+0000","last_became_active":"2024-01-16T15:00:03.578921+0000","last_became_peered":"2024-01-16T15:00:03.578921+0000","last_unstale":"2024-01-16T15:00:04.558033+0000","last_undegraded":"2024-01-16T15:00:04.558033+0000","last_fullsized":"2024-01-16T15:00:04.558033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:32:18.387639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.554616+0000","last_change":"2024-01-16T15:00:03.571130+0000","last_active":"2024-01-16T15:00:04.554616+0000","last_peered":"2024-01-16T15:00:04.554616+0000","last_clean":"2024-01-16T15:00:04.554616+0000","last_became_active":"2024-01-16T15:00:03.570816+0000","last_became_peered":"2024-01-16T15:00:03.570816+0000","last_unstale":"2024-01-16T15:00:04.554616+0000","last_undegraded":"2024-01-16T15:00:04.554616+0000","last_fullsized":"2024-01-16T15:00:04.554616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:06:24.858403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323971+0000","last_change":"2024-01-16T15:00:03.578171+0000","last_active":"2024-01-16T15:00:05.323971+0000","last_peered":"2024-01-16T15:00:05.323971+0000","last_clean":"2024-01-16T15:00:05.323971+0000","last_became_active":"2024-01-16T15:00:03.577849+0000","last_became_peered":"2024-01-16T15:00:03.577849+0000","last_unstale":"2024-01-16T15:00:05.323971+0000","last_undegraded":"2024-01-16T15:00:05.323971+0000","last_fullsized":"2024-01-16T15:00:05.323971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:11:11.328141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323719+0000","last_change":"2024-01-16T15:00:03.585019+0000","last_active":"2024-01-16T15:00:05.323719+0000","last_peered":"2024-01-16T15:00:05.323719+0000","last_clean":"2024-01-16T15:00:05.323719+0000","last_became_active":"2024-01-16T15:00:03.584747+0000","last_became_peered":"2024-01-16T15:00:03.584747+0000","last_unstale":"2024-01-16T15:00:05.323719+0000","last_undegraded":"2024-01-16T15:00:05.323719+0000","last_fullsized":"2024-01-16T15:00:05.323719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:44:37.235476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.556701+0000","last_change":"2024-01-16T15:00:03.577404+0000","last_active":"2024-01-16T15:00:04.556701+0000","last_peered":"2024-01-16T15:00:04.556701+0000","last_clean":"2024-01-16T15:00:04.556701+0000","last_became_active":"2024-01-16T15:00:03.577051+0000","last_became_peered":"2024-01-16T15:00:03.577051+0000","last_unstale":"2024-01-16T15:00:04.556701+0000","last_undegraded":"2024-01-16T15:00:04.556701+0000","last_fullsized":"2024-01-16T15:00:04.556701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:44:37.235476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.322548+0000","last_change":"2024-01-16T15:00:03.589065+0000","last_active":"2024-01-16T15:00:05.322548+0000","last_peered":"2024-01-16T15:00:05.322548+0000","last_clean":"2024-01-16T15:00:05.322548+0000","last_became_active":"2024-01-16T15:00:03.588776+0000","last_became_peered":"2024-01-16T15:00:03.588776+0000","last_unstale":"2024-01-16T15:00:05.322548+0000","last_undegraded":"2024-01-16T15:00:05.322548+0000","last_fullsized":"2024-01-16T15:00:05.322548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:06:24.858403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.556141+0000","last_change":"2024-01-16T15:00:03.570611+0000","last_active":"2024-01-16T15:00:04.556141+0000","last_peered":"2024-01-16T15:00:04.556141+0000","last_clean":"2024-01-16T15:00:04.556141+0000","last_became_active":"2024-01-16T15:00:03.569839+0000","last_became_peered":"2024-01-16T15:00:03.569839+0000","last_unstale":"2024-01-16T15:00:04.556141+0000","last_undegraded":"2024-01-16T15:00:04.556141+0000","last_fullsized":"2024-01-16T15:00:04.556141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:09:47.428018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.127040+0000","last_change":"2024-01-16T15:00:03.586954+0000","last_active":"2024-01-16T15:00:05.127040+0000","last_peered":"2024-01-16T15:00:05.127040+0000","last_clean":"2024-01-16T15:00:05.127040+0000","last_became_active":"2024-01-16T15:00:03.586663+0000","last_became_peered":"2024-01-16T15:00:03.586663+0000","last_unstale":"2024-01-16T15:00:05.127040+0000","last_undegraded":"2024-01-16T15:00:05.127040+0000","last_fullsized":"2024-01-16T15:00:05.127040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:34:39.229613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.558126+0000","last_change":"2024-01-16T15:00:03.596120+0000","last_active":"2024-01-16T15:00:04.558126+0000","last_peered":"2024-01-16T15:00:04.558126+0000","last_clean":"2024-01-16T15:00:04.558126+0000","last_became_active":"2024-01-16T15:00:03.595866+0000","last_became_peered":"2024-01-16T15:00:03.595866+0000","last_unstale":"2024-01-16T15:00:04.558126+0000","last_undegraded":"2024-01-16T15:00:04.558126+0000","last_fullsized":"2024-01-16T15:00:04.558126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:53:57.952288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.750726+0000","last_change":"2024-01-16T15:00:03.588019+0000","last_active":"2024-01-16T15:00:04.750726+0000","last_peered":"2024-01-16T15:00:04.750726+0000","last_clean":"2024-01-16T15:00:04.750726+0000","last_became_active":"2024-01-16T15:00:03.587359+0000","last_became_peered":"2024-01-16T15:00:03.587359+0000","last_unstale":"2024-01-16T15:00:04.750726+0000","last_undegraded":"2024-01-16T15:00:04.750726+0000","last_fullsized":"2024-01-16T15:00:04.750726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:31:56.335251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.750616+0000","last_change":"2024-01-16T15:00:03.590950+0000","last_active":"2024-01-16T15:00:04.750616+0000","last_peered":"2024-01-16T15:00:04.750616+0000","last_clean":"2024-01-16T15:00:04.750616+0000","last_became_active":"2024-01-16T15:00:03.590702+0000","last_became_peered":"2024-01-16T15:00:03.590702+0000","last_unstale":"2024-01-16T15:00:04.750616+0000","last_undegraded":"2024-01-16T15:00:04.750616+0000","last_fullsized":"2024-01-16T15:00:04.750616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:04:13.308774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.555500+0000","last_change":"2024-01-16T15:00:03.582929+0000","last_active":"2024-01-16T15:00:04.555500+0000","last_peered":"2024-01-16T15:00:04.555500+0000","last_clean":"2024-01-16T15:00:04.555500+0000","last_became_active":"2024-01-16T15:00:03.582040+0000","last_became_peered":"2024-01-16T15:00:03.582040+0000","last_unstale":"2024-01-16T15:00:04.555500+0000","last_undegraded":"2024-01-16T15:00:04.555500+0000","last_fullsized":"2024-01-16T15:00:04.555500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:10:55.291745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.127726+0000","last_change":"2024-01-16T15:00:03.581183+0000","last_active":"2024-01-16T15:00:05.127726+0000","last_peered":"2024-01-16T15:00:05.127726+0000","last_clean":"2024-01-16T15:00:05.127726+0000","last_became_active":"2024-01-16T15:00:03.578642+0000","last_became_peered":"2024-01-16T15:00:03.578642+0000","last_unstale":"2024-01-16T15:00:05.127726+0000","last_undegraded":"2024-01-16T15:00:05.127726+0000","last_fullsized":"2024-01-16T15:00:05.127726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:06:24.858403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323409+0000","last_change":"2024-01-16T15:00:03.570282+0000","last_active":"2024-01-16T15:00:05.323409+0000","last_peered":"2024-01-16T15:00:05.323409+0000","last_clean":"2024-01-16T15:00:05.323409+0000","last_became_active":"2024-01-16T15:00:03.569988+0000","last_became_peered":"2024-01-16T15:00:03.569988+0000","last_unstale":"2024-01-16T15:00:05.323409+0000","last_undegraded":"2024-01-16T15:00:05.323409+0000","last_fullsized":"2024-01-16T15:00:05.323409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:16:00.005822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.554056+0000","last_change":"2024-01-16T15:00:03.584705+0000","last_active":"2024-01-16T15:00:04.554056+0000","last_peered":"2024-01-16T15:00:04.554056+0000","last_clean":"2024-01-16T15:00:04.554056+0000","last_became_active":"2024-01-16T15:00:03.584361+0000","last_became_peered":"2024-01-16T15:00:03.584361+0000","last_unstale":"2024-01-16T15:00:04.554056+0000","last_undegraded":"2024-01-16T15:00:04.554056+0000","last_fullsized":"2024-01-16T15:00:04.554056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:11:11.328141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.322398+0000","last_change":"2024-01-16T15:00:03.587177+0000","last_active":"2024-01-16T15:00:05.322398+0000","last_peered":"2024-01-16T15:00:05.322398+0000","last_clean":"2024-01-16T15:00:05.322398+0000","last_became_active":"2024-01-16T15:00:03.586803+0000","last_became_peered":"2024-01-16T15:00:03.586803+0000","last_unstale":"2024-01-16T15:00:05.322398+0000","last_undegraded":"2024-01-16T15:00:05.322398+0000","last_fullsized":"2024-01-16T15:00:05.322398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:03:10.226190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323936+0000","last_change":"2024-01-16T15:00:03.595310+0000","last_active":"2024-01-16T15:00:05.323936+0000","last_peered":"2024-01-16T15:00:05.323936+0000","last_clean":"2024-01-16T15:00:05.323936+0000","last_became_active":"2024-01-16T15:00:03.589586+0000","last_became_peered":"2024-01-16T15:00:03.589586+0000","last_unstale":"2024-01-16T15:00:05.323936+0000","last_undegraded":"2024-01-16T15:00:05.323936+0000","last_fullsized":"2024-01-16T15:00:05.323936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:29:26.138536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.322465+0000","last_change":"2024-01-16T15:00:03.583181+0000","last_active":"2024-01-16T15:00:05.322465+0000","last_peered":"2024-01-16T15:00:05.322465+0000","last_clean":"2024-01-16T15:00:05.322465+0000","last_became_active":"2024-01-16T15:00:03.582404+0000","last_became_peered":"2024-01-16T15:00:03.582404+0000","last_unstale":"2024-01-16T15:00:05.322465+0000","last_undegraded":"2024-01-16T15:00:05.322465+0000","last_fullsized":"2024-01-16T15:00:05.322465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:25:28.863720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.557296+0000","last_change":"2024-01-16T15:00:03.577376+0000","last_active":"2024-01-16T15:00:04.557296+0000","last_peered":"2024-01-16T15:00:04.557296+0000","last_clean":"2024-01-16T15:00:04.557296+0000","last_became_active":"2024-01-16T15:00:03.577167+0000","last_became_peered":"2024-01-16T15:00:03.577167+0000","last_unstale":"2024-01-16T15:00:04.557296+0000","last_undegraded":"2024-01-16T15:00:04.557296+0000","last_fullsized":"2024-01-16T15:00:04.557296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:14:46.103689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.750577+0000","last_change":"2024-01-16T15:00:03.587122+0000","last_active":"2024-01-16T15:00:04.750577+0000","last_peered":"2024-01-16T15:00:04.750577+0000","last_clean":"2024-01-16T15:00:04.750577+0000","last_became_active":"2024-01-16T15:00:03.586244+0000","last_became_peered":"2024-01-16T15:00:03.586244+0000","last_unstale":"2024-01-16T15:00:04.750577+0000","last_undegraded":"2024-01-16T15:00:04.750577+0000","last_fullsized":"2024-01-16T15:00:04.750577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:57:14.928029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323320+0000","last_change":"2024-01-16T15:00:03.572186+0000","last_active":"2024-01-16T15:00:05.323320+0000","last_peered":"2024-01-16T15:00:05.323320+0000","last_clean":"2024-01-16T15:00:05.323320+0000","last_became_active":"2024-01-16T15:00:03.571795+0000","last_became_peered":"2024-01-16T15:00:03.571795+0000","last_unstale":"2024-01-16T15:00:05.323320+0000","last_undegraded":"2024-01-16T15:00:05.323320+0000","last_fullsized":"2024-01-16T15:00:05.323320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:18:11.185813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.322924+0000","last_change":"2024-01-16T15:00:03.589661+0000","last_active":"2024-01-16T15:00:05.322924+0000","last_peered":"2024-01-16T15:00:05.322924+0000","last_clean":"2024-01-16T15:00:05.322924+0000","last_became_active":"2024-01-16T15:00:03.589365+0000","last_became_peered":"2024-01-16T15:00:03.589365+0000","last_unstale":"2024-01-16T15:00:05.322924+0000","last_undegraded":"2024-01-16T15:00:05.322924+0000","last_fullsized":"2024-01-16T15:00:05.322924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:04:13.308774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.127047+0000","last_change":"2024-01-16T15:00:03.591522+0000","last_active":"2024-01-16T15:00:05.127047+0000","last_peered":"2024-01-16T15:00:05.127047+0000","last_clean":"2024-01-16T15:00:05.127047+0000","last_became_active":"2024-01-16T15:00:03.591143+0000","last_became_peered":"2024-01-16T15:00:03.591143+0000","last_unstale":"2024-01-16T15:00:05.127047+0000","last_undegraded":"2024-01-16T15:00:05.127047+0000","last_fullsized":"2024-01-16T15:00:05.127047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:11:11.328141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.322833+0000","last_change":"2024-01-16T15:00:03.585381+0000","last_active":"2024-01-16T15:00:05.322833+0000","last_peered":"2024-01-16T15:00:05.322833+0000","last_clean":"2024-01-16T15:00:05.322833+0000","last_became_active":"2024-01-16T15:00:03.584858+0000","last_became_peered":"2024-01-16T15:00:03.584858+0000","last_unstale":"2024-01-16T15:00:05.322833+0000","last_undegraded":"2024-01-16T15:00:05.322833+0000","last_fullsized":"2024-01-16T15:00:05.322833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:57:14.928029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.127981+0000","last_change":"2024-01-16T15:00:03.591627+0000","last_active":"2024-01-16T15:00:05.127981+0000","last_peered":"2024-01-16T15:00:05.127981+0000","last_clean":"2024-01-16T15:00:05.127981+0000","last_became_active":"2024-01-16T15:00:03.591216+0000","last_became_peered":"2024-01-16T15:00:03.591216+0000","last_unstale":"2024-01-16T15:00:05.127981+0000","last_undegraded":"2024-01-16T15:00:05.127981+0000","last_fullsized":"2024-01-16T15:00:05.127981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:31:56.335251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.750822+0000","last_change":"2024-01-16T15:00:03.588262+0000","last_active":"2024-01-16T15:00:04.750822+0000","last_peered":"2024-01-16T15:00:04.750822+0000","last_clean":"2024-01-16T15:00:04.750822+0000","last_became_active":"2024-01-16T15:00:03.587728+0000","last_became_peered":"2024-01-16T15:00:03.587728+0000","last_unstale":"2024-01-16T15:00:04.750822+0000","last_undegraded":"2024-01-16T15:00:04.750822+0000","last_fullsized":"2024-01-16T15:00:04.750822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:05:08.127179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.554267+0000","last_change":"2024-01-16T15:00:03.576892+0000","last_active":"2024-01-16T15:00:04.554267+0000","last_peered":"2024-01-16T15:00:04.554267+0000","last_clean":"2024-01-16T15:00:04.554267+0000","last_became_active":"2024-01-16T15:00:03.574566+0000","last_became_peered":"2024-01-16T15:00:03.574566+0000","last_unstale":"2024-01-16T15:00:04.554267+0000","last_undegraded":"2024-01-16T15:00:04.554267+0000","last_fullsized":"2024-01-16T15:00:04.554267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:33:59.253631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.127898+0000","last_change":"2024-01-16T15:00:03.591601+0000","last_active":"2024-01-16T15:00:05.127898+0000","last_peered":"2024-01-16T15:00:05.127898+0000","last_clean":"2024-01-16T15:00:05.127898+0000","last_became_active":"2024-01-16T15:00:03.591271+0000","last_became_peered":"2024-01-16T15:00:03.591271+0000","last_unstale":"2024-01-16T15:00:05.127898+0000","last_undegraded":"2024-01-16T15:00:05.127898+0000","last_fullsized":"2024-01-16T15:00:05.127898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:04:13.308774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323503+0000","last_change":"2024-01-16T15:00:03.580655+0000","last_active":"2024-01-16T15:00:05.323503+0000","last_peered":"2024-01-16T15:00:05.323503+0000","last_clean":"2024-01-16T15:00:05.323503+0000","last_became_active":"2024-01-16T15:00:03.580091+0000","last_became_peered":"2024-01-16T15:00:03.580091+0000","last_unstale":"2024-01-16T15:00:05.323503+0000","last_undegraded":"2024-01-16T15:00:05.323503+0000","last_fullsized":"2024-01-16T15:00:05.323503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:34:39.229613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.322959+0000","last_change":"2024-01-16T15:00:03.588486+0000","last_active":"2024-01-16T15:00:05.322959+0000","last_peered":"2024-01-16T15:00:05.322959+0000","last_clean":"2024-01-16T15:00:05.322959+0000","last_became_active":"2024-01-16T15:00:03.586447+0000","last_became_peered":"2024-01-16T15:00:03.586447+0000","last_unstale":"2024-01-16T15:00:05.322959+0000","last_undegraded":"2024-01-16T15:00:05.322959+0000","last_fullsized":"2024-01-16T15:00:05.322959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:31:56.335251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.322016+0000","last_change":"2024-01-16T15:00:03.581151+0000","last_active":"2024-01-16T15:00:05.322016+0000","last_peered":"2024-01-16T15:00:05.322016+0000","last_clean":"2024-01-16T15:00:05.322016+0000","last_became_active":"2024-01-16T15:00:03.579463+0000","last_became_peered":"2024-01-16T15:00:03.579463+0000","last_unstale":"2024-01-16T15:00:05.322016+0000","last_undegraded":"2024-01-16T15:00:05.322016+0000","last_fullsized":"2024-01-16T15:00:05.322016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:10:55.291745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.750064+0000","last_change":"2024-01-16T15:00:03.588682+0000","last_active":"2024-01-16T15:00:04.750064+0000","last_peered":"2024-01-16T15:00:04.750064+0000","last_clean":"2024-01-16T15:00:04.750064+0000","last_became_active":"2024-01-16T15:00:03.588148+0000","last_became_peered":"2024-01-16T15:00:03.588148+0000","last_unstale":"2024-01-16T15:00:04.750064+0000","last_undegraded":"2024-01-16T15:00:04.750064+0000","last_fullsized":"2024-01-16T15:00:04.750064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:33:59.253631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.555160+0000","last_change":"2024-01-16T15:00:03.580412+0000","last_active":"2024-01-16T15:00:04.555160+0000","last_peered":"2024-01-16T15:00:04.555160+0000","last_clean":"2024-01-16T15:00:04.555160+0000","last_became_active":"2024-01-16T15:00:03.572852+0000","last_became_peered":"2024-01-16T15:00:03.572852+0000","last_unstale":"2024-01-16T15:00:04.555160+0000","last_undegraded":"2024-01-16T15:00:04.555160+0000","last_fullsized":"2024-01-16T15:00:04.555160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:48:44.736531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.613598+0000","last_change":"2024-01-16T15:00:02.635074+0000","last_active":"2024-01-16T15:00:03.613598+0000","last_peered":"2024-01-16T15:00:03.613598+0000","last_clean":"2024-01-16T15:00:03.613598+0000","last_became_active":"2024-01-16T15:00:02.634767+0000","last_became_peered":"2024-01-16T15:00:02.634767+0000","last_unstale":"2024-01-16T15:00:03.613598+0000","last_undegraded":"2024-01-16T15:00:03.613598+0000","last_fullsized":"2024-01-16T15:00:03.613598+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:15:58.991139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.324292+0000","last_change":"2024-01-16T15:00:03.573187+0000","last_active":"2024-01-16T15:00:05.324292+0000","last_peered":"2024-01-16T15:00:05.324292+0000","last_clean":"2024-01-16T15:00:05.324292+0000","last_became_active":"2024-01-16T15:00:03.572755+0000","last_became_peered":"2024-01-16T15:00:03.572755+0000","last_unstale":"2024-01-16T15:00:05.324292+0000","last_undegraded":"2024-01-16T15:00:05.324292+0000","last_fullsized":"2024-01-16T15:00:05.324292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:06:24.858403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.610883+0000","last_change":"2024-01-16T15:00:02.632028+0000","last_active":"2024-01-16T15:00:03.610883+0000","last_peered":"2024-01-16T15:00:03.610883+0000","last_clean":"2024-01-16T15:00:03.610883+0000","last_became_active":"2024-01-16T15:00:02.626019+0000","last_became_peered":"2024-01-16T15:00:02.626019+0000","last_unstale":"2024-01-16T15:00:03.610883+0000","last_undegraded":"2024-01-16T15:00:03.610883+0000","last_fullsized":"2024-01-16T15:00:03.610883+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:48:43.721848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.568172+0000","last_change":"2024-01-16T15:00:02.635363+0000","last_active":"2024-01-16T15:00:03.568172+0000","last_peered":"2024-01-16T15:00:03.568172+0000","last_clean":"2024-01-16T15:00:03.568172+0000","last_became_active":"2024-01-16T15:00:02.634387+0000","last_became_peered":"2024-01-16T15:00:02.634387+0000","last_unstale":"2024-01-16T15:00:03.568172+0000","last_undegraded":"2024-01-16T15:00:03.568172+0000","last_fullsized":"2024-01-16T15:00:03.568172+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:10:54.277062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.127245+0000","last_change":"2024-01-16T15:00:03.576575+0000","last_active":"2024-01-16T15:00:05.127245+0000","last_peered":"2024-01-16T15:00:05.127245+0000","last_clean":"2024-01-16T15:00:05.127245+0000","last_became_active":"2024-01-16T15:00:03.576266+0000","last_became_peered":"2024-01-16T15:00:03.576266+0000","last_unstale":"2024-01-16T15:00:05.127245+0000","last_undegraded":"2024-01-16T15:00:05.127245+0000","last_fullsized":"2024-01-16T15:00:05.127245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:28:03.212278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.584305+0000","last_change":"2024-01-16T15:00:02.636608+0000","last_active":"2024-01-16T15:00:03.584305+0000","last_peered":"2024-01-16T15:00:03.584305+0000","last_clean":"2024-01-16T15:00:03.584305+0000","last_became_active":"2024-01-16T15:00:02.636235+0000","last_became_peered":"2024-01-16T15:00:02.636235+0000","last_unstale":"2024-01-16T15:00:03.584305+0000","last_undegraded":"2024-01-16T15:00:03.584305+0000","last_fullsized":"2024-01-16T15:00:03.584305+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:11:10.313458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.553948+0000","last_change":"2024-01-16T15:00:03.579753+0000","last_active":"2024-01-16T15:00:04.553948+0000","last_peered":"2024-01-16T15:00:04.553948+0000","last_clean":"2024-01-16T15:00:04.553948+0000","last_became_active":"2024-01-16T15:00:03.579415+0000","last_became_peered":"2024-01-16T15:00:03.579415+0000","last_unstale":"2024-01-16T15:00:04.553948+0000","last_undegraded":"2024-01-16T15:00:04.553948+0000","last_fullsized":"2024-01-16T15:00:04.553948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:34:39.229613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.572357+0000","last_change":"2024-01-16T15:00:02.635549+0000","last_active":"2024-01-16T15:00:03.572357+0000","last_peered":"2024-01-16T15:00:03.572357+0000","last_clean":"2024-01-16T15:00:03.572357+0000","last_became_active":"2024-01-16T15:00:02.635309+0000","last_became_peered":"2024-01-16T15:00:02.635309+0000","last_unstale":"2024-01-16T15:00:03.572357+0000","last_undegraded":"2024-01-16T15:00:03.572357+0000","last_fullsized":"2024-01-16T15:00:03.572357+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:34:38.214930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323817+0000","last_change":"2024-01-16T15:00:03.586931+0000","last_active":"2024-01-16T15:00:05.323817+0000","last_peered":"2024-01-16T15:00:05.323817+0000","last_clean":"2024-01-16T15:00:05.323817+0000","last_became_active":"2024-01-16T15:00:03.586531+0000","last_became_peered":"2024-01-16T15:00:03.586531+0000","last_unstale":"2024-01-16T15:00:05.323817+0000","last_undegraded":"2024-01-16T15:00:05.323817+0000","last_fullsized":"2024-01-16T15:00:05.323817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:18:16.333187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.583406+0000","last_change":"2024-01-16T15:00:02.638637+0000","last_active":"2024-01-16T15:00:03.583406+0000","last_peered":"2024-01-16T15:00:03.583406+0000","last_clean":"2024-01-16T15:00:03.583406+0000","last_became_active":"2024-01-16T15:00:02.638355+0000","last_became_peered":"2024-01-16T15:00:02.638355+0000","last_unstale":"2024-01-16T15:00:03.583406+0000","last_undegraded":"2024-01-16T15:00:03.583406+0000","last_fullsized":"2024-01-16T15:00:03.583406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:53:48.731286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.324191+0000","last_change":"2024-01-16T15:00:03.587105+0000","last_active":"2024-01-16T15:00:05.324191+0000","last_peered":"2024-01-16T15:00:05.324191+0000","last_clean":"2024-01-16T15:00:05.324191+0000","last_became_active":"2024-01-16T15:00:03.586572+0000","last_became_peered":"2024-01-16T15:00:03.586572+0000","last_unstale":"2024-01-16T15:00:05.324191+0000","last_undegraded":"2024-01-16T15:00:05.324191+0000","last_fullsized":"2024-01-16T15:00:05.324191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:03:10.226190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.611665+0000","last_change":"2024-01-16T15:00:02.636352+0000","last_active":"2024-01-16T15:00:03.611665+0000","last_peered":"2024-01-16T15:00:03.611665+0000","last_clean":"2024-01-16T15:00:03.611665+0000","last_became_active":"2024-01-16T15:00:02.635069+0000","last_became_peered":"2024-01-16T15:00:02.635069+0000","last_unstale":"2024-01-16T15:00:03.611665+0000","last_undegraded":"2024-01-16T15:00:03.611665+0000","last_fullsized":"2024-01-16T15:00:03.611665+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:10:54.277062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.555540+0000","last_change":"2024-01-16T15:00:03.580722+0000","last_active":"2024-01-16T15:00:04.555540+0000","last_peered":"2024-01-16T15:00:04.555540+0000","last_clean":"2024-01-16T15:00:04.555540+0000","last_became_active":"2024-01-16T15:00:03.572307+0000","last_became_peered":"2024-01-16T15:00:03.572307+0000","last_unstale":"2024-01-16T15:00:04.555540+0000","last_undegraded":"2024-01-16T15:00:04.555540+0000","last_fullsized":"2024-01-16T15:00:04.555540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:25:28.863720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.617703+0000","last_change":"2024-01-16T15:00:02.625026+0000","last_active":"2024-01-16T15:00:03.617703+0000","last_peered":"2024-01-16T15:00:03.617703+0000","last_clean":"2024-01-16T15:00:03.617703+0000","last_became_active":"2024-01-16T15:00:02.622889+0000","last_became_peered":"2024-01-16T15:00:02.622889+0000","last_unstale":"2024-01-16T15:00:03.617703+0000","last_undegraded":"2024-01-16T15:00:03.617703+0000","last_fullsized":"2024-01-16T15:00:03.617703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:15:58.991139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.557063+0000","last_change":"2024-01-16T15:00:03.581190+0000","last_active":"2024-01-16T15:00:04.557063+0000","last_peered":"2024-01-16T15:00:04.557063+0000","last_clean":"2024-01-16T15:00:04.557063+0000","last_became_active":"2024-01-16T15:00:03.580903+0000","last_became_peered":"2024-01-16T15:00:03.580903+0000","last_unstale":"2024-01-16T15:00:04.557063+0000","last_undegraded":"2024-01-16T15:00:04.557063+0000","last_fullsized":"2024-01-16T15:00:04.557063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:17:11.329531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.615038+0000","last_change":"2024-01-16T15:00:02.622318+0000","last_active":"2024-01-16T15:00:03.615038+0000","last_peered":"2024-01-16T15:00:03.615038+0000","last_clean":"2024-01-16T15:00:03.615038+0000","last_became_active":"2024-01-16T15:00:02.622049+0000","last_became_peered":"2024-01-16T15:00:02.622049+0000","last_unstale":"2024-01-16T15:00:03.615038+0000","last_undegraded":"2024-01-16T15:00:03.615038+0000","last_fullsized":"2024-01-16T15:00:03.615038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:11:22.855828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323468+0000","last_change":"2024-01-16T15:00:03.574905+0000","last_active":"2024-01-16T15:00:05.323468+0000","last_peered":"2024-01-16T15:00:05.323468+0000","last_clean":"2024-01-16T15:00:05.323468+0000","last_became_active":"2024-01-16T15:00:03.572077+0000","last_became_peered":"2024-01-16T15:00:03.572077+0000","last_unstale":"2024-01-16T15:00:05.323468+0000","last_undegraded":"2024-01-16T15:00:05.323468+0000","last_fullsized":"2024-01-16T15:00:05.323468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:53:49.745969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.583051+0000","last_change":"2024-01-16T15:00:02.632349+0000","last_active":"2024-01-16T15:00:03.583051+0000","last_peered":"2024-01-16T15:00:03.583051+0000","last_clean":"2024-01-16T15:00:03.583051+0000","last_became_active":"2024-01-16T15:00:02.632038+0000","last_became_peered":"2024-01-16T15:00:02.632038+0000","last_unstale":"2024-01-16T15:00:03.583051+0000","last_undegraded":"2024-01-16T15:00:03.583051+0000","last_fullsized":"2024-01-16T15:00:03.583051+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:34:38.214930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.556735+0000","last_change":"2024-01-16T15:00:03.584616+0000","last_active":"2024-01-16T15:00:04.556735+0000","last_peered":"2024-01-16T15:00:04.556735+0000","last_clean":"2024-01-16T15:00:04.556735+0000","last_became_active":"2024-01-16T15:00:03.584348+0000","last_became_peered":"2024-01-16T15:00:03.584348+0000","last_unstale":"2024-01-16T15:00:04.556735+0000","last_undegraded":"2024-01-16T15:00:04.556735+0000","last_fullsized":"2024-01-16T15:00:04.556735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:18:16.333187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.569602+0000","last_change":"2024-01-16T15:00:02.628258+0000","last_active":"2024-01-16T15:00:03.569602+0000","last_peered":"2024-01-16T15:00:03.569602+0000","last_clean":"2024-01-16T15:00:03.569602+0000","last_became_active":"2024-01-16T15:00:02.627972+0000","last_became_peered":"2024-01-16T15:00:02.627972+0000","last_unstale":"2024-01-16T15:00:03.569602+0000","last_undegraded":"2024-01-16T15:00:03.569602+0000","last_fullsized":"2024-01-16T15:00:03.569602+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:25:27.849037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.750102+0000","last_change":"2024-01-16T15:00:03.588418+0000","last_active":"2024-01-16T15:00:04.750102+0000","last_peered":"2024-01-16T15:00:04.750102+0000","last_clean":"2024-01-16T15:00:04.750102+0000","last_became_active":"2024-01-16T15:00:03.587735+0000","last_became_peered":"2024-01-16T15:00:03.587735+0000","last_unstale":"2024-01-16T15:00:04.750102+0000","last_undegraded":"2024-01-16T15:00:04.750102+0000","last_fullsized":"2024-01-16T15:00:04.750102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:03:10.226190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.617567+0000","last_change":"2024-01-16T15:00:02.616381+0000","last_active":"2024-01-16T15:00:03.617567+0000","last_peered":"2024-01-16T15:00:03.617567+0000","last_clean":"2024-01-16T15:00:03.617567+0000","last_became_active":"2024-01-16T15:00:02.615244+0000","last_became_peered":"2024-01-16T15:00:02.615244+0000","last_unstale":"2024-01-16T15:00:03.617567+0000","last_undegraded":"2024-01-16T15:00:03.617567+0000","last_fullsized":"2024-01-16T15:00:03.617567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:21:47.228684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.556253+0000","last_change":"2024-01-16T15:00:03.584478+0000","last_active":"2024-01-16T15:00:04.556253+0000","last_peered":"2024-01-16T15:00:04.556253+0000","last_clean":"2024-01-16T15:00:04.556253+0000","last_became_active":"2024-01-16T15:00:03.583891+0000","last_became_peered":"2024-01-16T15:00:03.583891+0000","last_unstale":"2024-01-16T15:00:04.556253+0000","last_undegraded":"2024-01-16T15:00:04.556253+0000","last_fullsized":"2024-01-16T15:00:04.556253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:02:58.710795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.609993+0000","last_change":"2024-01-16T15:00:02.623236+0000","last_active":"2024-01-16T15:00:03.609993+0000","last_peered":"2024-01-16T15:00:03.609993+0000","last_clean":"2024-01-16T15:00:03.609993+0000","last_became_active":"2024-01-16T15:00:02.619961+0000","last_became_peered":"2024-01-16T15:00:02.619961+0000","last_unstale":"2024-01-16T15:00:03.609993+0000","last_undegraded":"2024-01-16T15:00:03.609993+0000","last_fullsized":"2024-01-16T15:00:03.609993+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:15:08.623284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.557756+0000","last_change":"2024-01-16T15:00:03.584135+0000","last_active":"2024-01-16T15:00:04.557756+0000","last_peered":"2024-01-16T15:00:04.557756+0000","last_clean":"2024-01-16T15:00:04.557756+0000","last_became_active":"2024-01-16T15:00:03.583742+0000","last_became_peered":"2024-01-16T15:00:03.583742+0000","last_unstale":"2024-01-16T15:00:04.557756+0000","last_undegraded":"2024-01-16T15:00:04.557756+0000","last_fullsized":"2024-01-16T15:00:04.557756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:54:11.712679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.613078+0000","last_change":"2024-01-16T15:00:02.625540+0000","last_active":"2024-01-16T15:00:03.613078+0000","last_peered":"2024-01-16T15:00:03.613078+0000","last_clean":"2024-01-16T15:00:03.613078+0000","last_became_active":"2024-01-16T15:00:02.625192+0000","last_became_peered":"2024-01-16T15:00:02.625192+0000","last_unstale":"2024-01-16T15:00:03.613078+0000","last_undegraded":"2024-01-16T15:00:03.613078+0000","last_fullsized":"2024-01-16T15:00:03.613078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:48:43.721848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.558230+0000","last_change":"2024-01-16T15:00:03.591505+0000","last_active":"2024-01-16T15:00:04.558230+0000","last_peered":"2024-01-16T15:00:04.558230+0000","last_clean":"2024-01-16T15:00:04.558230+0000","last_became_active":"2024-01-16T15:00:03.591187+0000","last_became_peered":"2024-01-16T15:00:03.591187+0000","last_unstale":"2024-01-16T15:00:04.558230+0000","last_undegraded":"2024-01-16T15:00:04.558230+0000","last_fullsized":"2024-01-16T15:00:04.558230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:05:33.003372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.610948+0000","last_change":"2024-01-16T15:00:02.627127+0000","last_active":"2024-01-16T15:00:03.610948+0000","last_peered":"2024-01-16T15:00:03.610948+0000","last_clean":"2024-01-16T15:00:03.610948+0000","last_became_active":"2024-01-16T15:00:02.624007+0000","last_became_peered":"2024-01-16T15:00:02.624007+0000","last_unstale":"2024-01-16T15:00:03.610948+0000","last_undegraded":"2024-01-16T15:00:03.610948+0000","last_fullsized":"2024-01-16T15:00:03.610948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:18:10.171130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323506+0000","last_change":"2024-01-16T15:00:03.588356+0000","last_active":"2024-01-16T15:00:05.323506+0000","last_peered":"2024-01-16T15:00:05.323506+0000","last_clean":"2024-01-16T15:00:05.323506+0000","last_became_active":"2024-01-16T15:00:03.588094+0000","last_became_peered":"2024-01-16T15:00:03.588094+0000","last_unstale":"2024-01-16T15:00:05.323506+0000","last_undegraded":"2024-01-16T15:00:05.323506+0000","last_fullsized":"2024-01-16T15:00:05.323506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:02:58.710795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.571707+0000","last_change":"2024-01-16T15:00:02.628534+0000","last_active":"2024-01-16T15:00:03.571707+0000","last_peered":"2024-01-16T15:00:03.571707+0000","last_clean":"2024-01-16T15:00:03.571707+0000","last_became_active":"2024-01-16T15:00:02.627252+0000","last_became_peered":"2024-01-16T15:00:02.627252+0000","last_unstale":"2024-01-16T15:00:03.571707+0000","last_undegraded":"2024-01-16T15:00:03.571707+0000","last_fullsized":"2024-01-16T15:00:03.571707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:17:47.960980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.556971+0000","last_change":"2024-01-16T15:00:03.588973+0000","last_active":"2024-01-16T15:00:04.556971+0000","last_peered":"2024-01-16T15:00:04.556971+0000","last_clean":"2024-01-16T15:00:04.556971+0000","last_became_active":"2024-01-16T15:00:03.588673+0000","last_became_peered":"2024-01-16T15:00:03.588673+0000","last_unstale":"2024-01-16T15:00:04.556971+0000","last_undegraded":"2024-01-16T15:00:04.556971+0000","last_fullsized":"2024-01-16T15:00:04.556971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:29:26.138536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.554975+0000","last_change":"2024-01-16T14:59:53.430715+0000","last_active":"2024-01-16T15:00:03.554975+0000","last_peered":"2024-01-16T15:00:03.554975+0000","last_clean":"2024-01-16T15:00:03.554975+0000","last_became_active":"2024-01-16T14:59:53.428209+0000","last_became_peered":"2024-01-16T14:59:53.428209+0000","last_unstale":"2024-01-16T15:00:03.554975+0000","last_undegraded":"2024-01-16T15:00:03.554975+0000","last_fullsized":"2024-01-16T15:00:03.554975+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:59:52.011232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:59:52.011232+0000","last_clean_scrub_stamp":"2024-01-16T14:59:52.011232+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:57:42.594848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.585945+0000","last_change":"2024-01-16T15:00:02.635313+0000","last_active":"2024-01-16T15:00:03.585945+0000","last_peered":"2024-01-16T15:00:03.585945+0000","last_clean":"2024-01-16T15:00:03.585945+0000","last_became_active":"2024-01-16T15:00:02.634804+0000","last_became_peered":"2024-01-16T15:00:02.634804+0000","last_unstale":"2024-01-16T15:00:03.585945+0000","last_undegraded":"2024-01-16T15:00:03.585945+0000","last_fullsized":"2024-01-16T15:00:03.585945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:28:02.197595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.127292+0000","last_change":"2024-01-16T15:00:03.590575+0000","last_active":"2024-01-16T15:00:05.127292+0000","last_peered":"2024-01-16T15:00:05.127292+0000","last_clean":"2024-01-16T15:00:05.127292+0000","last_became_active":"2024-01-16T15:00:03.590218+0000","last_became_peered":"2024-01-16T15:00:03.590218+0000","last_unstale":"2024-01-16T15:00:05.127292+0000","last_undegraded":"2024-01-16T15:00:05.127292+0000","last_fullsized":"2024-01-16T15:00:05.127292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:33:59.253631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.555195+0000","last_change":"2024-01-16T15:00:02.636675+0000","last_active":"2024-01-16T15:00:03.555195+0000","last_peered":"2024-01-16T15:00:03.555195+0000","last_clean":"2024-01-16T15:00:03.555195+0000","last_became_active":"2024-01-16T15:00:02.633102+0000","last_became_peered":"2024-01-16T15:00:02.633102+0000","last_unstale":"2024-01-16T15:00:03.555195+0000","last_undegraded":"2024-01-16T15:00:03.555195+0000","last_fullsized":"2024-01-16T15:00:03.555195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:57:13.913346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323042+0000","last_change":"2024-01-16T15:00:03.583866+0000","last_active":"2024-01-16T15:00:05.323042+0000","last_peered":"2024-01-16T15:00:05.323042+0000","last_clean":"2024-01-16T15:00:05.323042+0000","last_became_active":"2024-01-16T15:00:03.583561+0000","last_became_peered":"2024-01-16T15:00:03.583561+0000","last_unstale":"2024-01-16T15:00:05.323042+0000","last_undegraded":"2024-01-16T15:00:05.323042+0000","last_fullsized":"2024-01-16T15:00:05.323042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:25:28.863720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.571916+0000","last_change":"2024-01-16T15:00:02.624133+0000","last_active":"2024-01-16T15:00:03.571916+0000","last_peered":"2024-01-16T15:00:03.571916+0000","last_clean":"2024-01-16T15:00:03.571916+0000","last_became_active":"2024-01-16T15:00:02.623883+0000","last_became_peered":"2024-01-16T15:00:02.623883+0000","last_unstale":"2024-01-16T15:00:03.571916+0000","last_undegraded":"2024-01-16T15:00:03.571916+0000","last_fullsized":"2024-01-16T15:00:03.571916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:00:52.721183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.554313+0000","last_change":"2024-01-16T15:00:03.571151+0000","last_active":"2024-01-16T15:00:04.554313+0000","last_peered":"2024-01-16T15:00:04.554313+0000","last_clean":"2024-01-16T15:00:04.554313+0000","last_became_active":"2024-01-16T15:00:03.570800+0000","last_became_peered":"2024-01-16T15:00:03.570800+0000","last_unstale":"2024-01-16T15:00:04.554313+0000","last_undegraded":"2024-01-16T15:00:04.554313+0000","last_fullsized":"2024-01-16T15:00:04.554313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:03:10.226190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.586154+0000","last_change":"2024-01-16T15:00:02.637246+0000","last_active":"2024-01-16T15:00:03.586154+0000","last_peered":"2024-01-16T15:00:03.586154+0000","last_clean":"2024-01-16T15:00:03.586154+0000","last_became_active":"2024-01-16T15:00:02.636465+0000","last_became_peered":"2024-01-16T15:00:02.636465+0000","last_unstale":"2024-01-16T15:00:03.586154+0000","last_undegraded":"2024-01-16T15:00:03.586154+0000","last_fullsized":"2024-01-16T15:00:03.586154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:33:58.238948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.555119+0000","last_change":"2024-01-16T15:00:03.580644+0000","last_active":"2024-01-16T15:00:04.555119+0000","last_peered":"2024-01-16T15:00:04.555119+0000","last_clean":"2024-01-16T15:00:04.555119+0000","last_became_active":"2024-01-16T15:00:03.573120+0000","last_became_peered":"2024-01-16T15:00:03.573120+0000","last_unstale":"2024-01-16T15:00:04.555119+0000","last_undegraded":"2024-01-16T15:00:04.555119+0000","last_fullsized":"2024-01-16T15:00:04.555119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:21:48.243367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.563287+0000","last_change":"2024-01-16T15:00:02.620390+0000","last_active":"2024-01-16T15:00:03.563287+0000","last_peered":"2024-01-16T15:00:03.563287+0000","last_clean":"2024-01-16T15:00:03.563287+0000","last_became_active":"2024-01-16T15:00:02.619046+0000","last_became_peered":"2024-01-16T15:00:02.619046+0000","last_unstale":"2024-01-16T15:00:03.563287+0000","last_undegraded":"2024-01-16T15:00:03.563287+0000","last_fullsized":"2024-01-16T15:00:03.563287+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:48:43.721848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323450+0000","last_change":"2024-01-16T15:00:03.587239+0000","last_active":"2024-01-16T15:00:05.323450+0000","last_peered":"2024-01-16T15:00:05.323450+0000","last_clean":"2024-01-16T15:00:05.323450+0000","last_became_active":"2024-01-16T15:00:03.586925+0000","last_became_peered":"2024-01-16T15:00:03.586925+0000","last_unstale":"2024-01-16T15:00:05.323450+0000","last_undegraded":"2024-01-16T15:00:05.323450+0000","last_fullsized":"2024-01-16T15:00:05.323450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:09:47.428018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.555099+0000","last_change":"2024-01-16T15:00:02.640889+0000","last_active":"2024-01-16T15:00:03.555099+0000","last_peered":"2024-01-16T15:00:03.555099+0000","last_clean":"2024-01-16T15:00:03.555099+0000","last_became_active":"2024-01-16T15:00:02.633676+0000","last_became_peered":"2024-01-16T15:00:02.633676+0000","last_unstale":"2024-01-16T15:00:03.555099+0000","last_undegraded":"2024-01-16T15:00:03.555099+0000","last_fullsized":"2024-01-16T15:00:03.555099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:03:09.211508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.554126+0000","last_change":"2024-01-16T15:00:03.576742+0000","last_active":"2024-01-16T15:00:04.554126+0000","last_peered":"2024-01-16T15:00:04.554126+0000","last_clean":"2024-01-16T15:00:04.554126+0000","last_became_active":"2024-01-16T15:00:03.576104+0000","last_became_peered":"2024-01-16T15:00:03.576104+0000","last_unstale":"2024-01-16T15:00:04.554126+0000","last_undegraded":"2024-01-16T15:00:04.554126+0000","last_fullsized":"2024-01-16T15:00:04.554126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:28:03.212278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.564812+0000","last_change":"2024-01-16T15:00:02.633317+0000","last_active":"2024-01-16T15:00:03.564812+0000","last_peered":"2024-01-16T15:00:03.564812+0000","last_clean":"2024-01-16T15:00:03.564812+0000","last_became_active":"2024-01-16T15:00:02.632996+0000","last_became_peered":"2024-01-16T15:00:02.632996+0000","last_unstale":"2024-01-16T15:00:03.564812+0000","last_undegraded":"2024-01-16T15:00:03.564812+0000","last_fullsized":"2024-01-16T15:00:03.564812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:53:48.731286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.128153+0000","last_change":"2024-01-16T15:00:03.577601+0000","last_active":"2024-01-16T15:00:05.128153+0000","last_peered":"2024-01-16T15:00:05.128153+0000","last_clean":"2024-01-16T15:00:05.128153+0000","last_became_active":"2024-01-16T15:00:03.577107+0000","last_became_peered":"2024-01-16T15:00:03.577107+0000","last_unstale":"2024-01-16T15:00:05.128153+0000","last_undegraded":"2024-01-16T15:00:05.128153+0000","last_fullsized":"2024-01-16T15:00:05.128153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:31:09.422997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.612223+0000","last_change":"2024-01-16T15:00:02.634633+0000","last_active":"2024-01-16T15:00:03.612223+0000","last_peered":"2024-01-16T15:00:03.612223+0000","last_clean":"2024-01-16T15:00:03.612223+0000","last_became_active":"2024-01-16T15:00:02.634372+0000","last_became_peered":"2024-01-16T15:00:02.634372+0000","last_unstale":"2024-01-16T15:00:03.612223+0000","last_undegraded":"2024-01-16T15:00:03.612223+0000","last_fullsized":"2024-01-16T15:00:03.612223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:11:10.313458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.324092+0000","last_change":"2024-01-16T15:00:03.575017+0000","last_active":"2024-01-16T15:00:05.324092+0000","last_peered":"2024-01-16T15:00:05.324092+0000","last_clean":"2024-01-16T15:00:05.324092+0000","last_became_active":"2024-01-16T15:00:03.573750+0000","last_became_peered":"2024-01-16T15:00:03.573750+0000","last_unstale":"2024-01-16T15:00:05.324092+0000","last_undegraded":"2024-01-16T15:00:05.324092+0000","last_fullsized":"2024-01-16T15:00:05.324092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:28:03.212278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.552344+0000","last_change":"2024-01-16T15:00:02.636963+0000","last_active":"2024-01-16T15:00:03.552344+0000","last_peered":"2024-01-16T15:00:03.552344+0000","last_clean":"2024-01-16T15:00:03.552344+0000","last_became_active":"2024-01-16T15:00:02.632840+0000","last_became_peered":"2024-01-16T15:00:02.632840+0000","last_unstale":"2024-01-16T15:00:03.552344+0000","last_undegraded":"2024-01-16T15:00:03.552344+0000","last_fullsized":"2024-01-16T15:00:03.552344+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:21:47.228684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.554498+0000","last_change":"2024-01-16T15:00:03.582851+0000","last_active":"2024-01-16T15:00:04.554498+0000","last_peered":"2024-01-16T15:00:04.554498+0000","last_clean":"2024-01-16T15:00:04.554498+0000","last_became_active":"2024-01-16T15:00:03.581868+0000","last_became_peered":"2024-01-16T15:00:03.581868+0000","last_unstale":"2024-01-16T15:00:04.554498+0000","last_undegraded":"2024-01-16T15:00:04.554498+0000","last_fullsized":"2024-01-16T15:00:04.554498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:00:53.735866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.563577+0000","last_change":"2024-01-16T15:00:02.622029+0000","last_active":"2024-01-16T15:00:03.563577+0000","last_peered":"2024-01-16T15:00:03.563577+0000","last_clean":"2024-01-16T15:00:03.563577+0000","last_became_active":"2024-01-16T15:00:02.620427+0000","last_became_peered":"2024-01-16T15:00:02.620427+0000","last_unstale":"2024-01-16T15:00:03.563577+0000","last_undegraded":"2024-01-16T15:00:03.563577+0000","last_fullsized":"2024-01-16T15:00:03.563577+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:25:27.849037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.127523+0000","last_change":"2024-01-16T15:00:03.580221+0000","last_active":"2024-01-16T15:00:05.127523+0000","last_peered":"2024-01-16T15:00:05.127523+0000","last_clean":"2024-01-16T15:00:05.127523+0000","last_became_active":"2024-01-16T15:00:03.579960+0000","last_became_peered":"2024-01-16T15:00:03.579960+0000","last_unstale":"2024-01-16T15:00:05.127523+0000","last_undegraded":"2024-01-16T15:00:05.127523+0000","last_fullsized":"2024-01-16T15:00:05.127523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:03:10.226190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.564091+0000","last_change":"2024-01-16T15:00:02.631776+0000","last_active":"2024-01-16T15:00:03.564091+0000","last_peered":"2024-01-16T15:00:03.564091+0000","last_clean":"2024-01-16T15:00:03.564091+0000","last_became_active":"2024-01-16T15:00:02.630658+0000","last_became_peered":"2024-01-16T15:00:02.630658+0000","last_unstale":"2024-01-16T15:00:03.564091+0000","last_undegraded":"2024-01-16T15:00:03.564091+0000","last_fullsized":"2024-01-16T15:00:03.564091+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:15:58.991139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.128076+0000","last_change":"2024-01-16T15:00:03.574061+0000","last_active":"2024-01-16T15:00:05.128076+0000","last_peered":"2024-01-16T15:00:05.128076+0000","last_clean":"2024-01-16T15:00:05.128076+0000","last_became_active":"2024-01-16T15:00:03.573691+0000","last_became_peered":"2024-01-16T15:00:03.573691+0000","last_unstale":"2024-01-16T15:00:05.128076+0000","last_undegraded":"2024-01-16T15:00:05.128076+0000","last_fullsized":"2024-01-16T15:00:05.128076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:05:08.127179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.615335+0000","last_change":"2024-01-16T15:00:02.634122+0000","last_active":"2024-01-16T15:00:03.615335+0000","last_peered":"2024-01-16T15:00:03.615335+0000","last_clean":"2024-01-16T15:00:03.615335+0000","last_became_active":"2024-01-16T15:00:02.633819+0000","last_became_peered":"2024-01-16T15:00:02.633819+0000","last_unstale":"2024-01-16T15:00:03.615335+0000","last_undegraded":"2024-01-16T15:00:03.615335+0000","last_fullsized":"2024-01-16T15:00:03.615335+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:53:48.731286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.750129+0000","last_change":"2024-01-16T15:00:03.590510+0000","last_active":"2024-01-16T15:00:04.750129+0000","last_peered":"2024-01-16T15:00:04.750129+0000","last_clean":"2024-01-16T15:00:04.750129+0000","last_became_active":"2024-01-16T15:00:03.588555+0000","last_became_peered":"2024-01-16T15:00:03.588555+0000","last_unstale":"2024-01-16T15:00:04.750129+0000","last_undegraded":"2024-01-16T15:00:04.750129+0000","last_fullsized":"2024-01-16T15:00:04.750129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:06:24.858403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323387+0000","last_change":"2024-01-16T15:00:03.594894+0000","last_active":"2024-01-16T15:00:05.323387+0000","last_peered":"2024-01-16T15:00:05.323387+0000","last_clean":"2024-01-16T15:00:05.323387+0000","last_became_active":"2024-01-16T15:00:03.592616+0000","last_became_peered":"2024-01-16T15:00:03.592616+0000","last_unstale":"2024-01-16T15:00:05.323387+0000","last_undegraded":"2024-01-16T15:00:05.323387+0000","last_fullsized":"2024-01-16T15:00:05.323387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:31:09.422997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.615281+0000","last_change":"2024-01-16T15:00:02.631757+0000","last_active":"2024-01-16T15:00:03.615281+0000","last_peered":"2024-01-16T15:00:03.615281+0000","last_clean":"2024-01-16T15:00:03.615281+0000","last_became_active":"2024-01-16T15:00:02.630946+0000","last_became_peered":"2024-01-16T15:00:02.630946+0000","last_unstale":"2024-01-16T15:00:03.615281+0000","last_undegraded":"2024-01-16T15:00:03.615281+0000","last_fullsized":"2024-01-16T15:00:03.615281+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:34:38.214930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323082+0000","last_change":"2024-01-16T15:00:03.593210+0000","last_active":"2024-01-16T15:00:05.323082+0000","last_peered":"2024-01-16T15:00:05.323082+0000","last_clean":"2024-01-16T15:00:05.323082+0000","last_became_active":"2024-01-16T15:00:03.592630+0000","last_became_peered":"2024-01-16T15:00:03.592630+0000","last_unstale":"2024-01-16T15:00:05.323082+0000","last_undegraded":"2024-01-16T15:00:05.323082+0000","last_fullsized":"2024-01-16T15:00:05.323082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:05:08.127179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.615363+0000","last_change":"2024-01-16T15:00:02.641351+0000","last_active":"2024-01-16T15:00:03.615363+0000","last_peered":"2024-01-16T15:00:03.615363+0000","last_clean":"2024-01-16T15:00:03.615363+0000","last_became_active":"2024-01-16T15:00:02.641021+0000","last_became_peered":"2024-01-16T15:00:02.641021+0000","last_unstale":"2024-01-16T15:00:03.615363+0000","last_undegraded":"2024-01-16T15:00:03.615363+0000","last_fullsized":"2024-01-16T15:00:03.615363+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:11:10.313458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.324164+0000","last_change":"2024-01-16T15:00:03.596490+0000","last_active":"2024-01-16T15:00:05.324164+0000","last_peered":"2024-01-16T15:00:05.324164+0000","last_clean":"2024-01-16T15:00:05.324164+0000","last_became_active":"2024-01-16T15:00:03.596137+0000","last_became_peered":"2024-01-16T15:00:03.596137+0000","last_unstale":"2024-01-16T15:00:05.324164+0000","last_undegraded":"2024-01-16T15:00:05.324164+0000","last_fullsized":"2024-01-16T15:00:05.324164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:33:59.253631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.612476+0000","last_change":"2024-01-16T15:00:02.630103+0000","last_active":"2024-01-16T15:00:03.612476+0000","last_peered":"2024-01-16T15:00:03.612476+0000","last_clean":"2024-01-16T15:00:03.612476+0000","last_became_active":"2024-01-16T15:00:02.629827+0000","last_became_peered":"2024-01-16T15:00:02.629827+0000","last_unstale":"2024-01-16T15:00:03.612476+0000","last_undegraded":"2024-01-16T15:00:03.612476+0000","last_fullsized":"2024-01-16T15:00:03.612476+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:25:27.849037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.127755+0000","last_change":"2024-01-16T15:00:03.583878+0000","last_active":"2024-01-16T15:00:05.127755+0000","last_peered":"2024-01-16T15:00:05.127755+0000","last_clean":"2024-01-16T15:00:05.127755+0000","last_became_active":"2024-01-16T15:00:03.583033+0000","last_became_peered":"2024-01-16T15:00:03.583033+0000","last_unstale":"2024-01-16T15:00:05.127755+0000","last_undegraded":"2024-01-16T15:00:05.127755+0000","last_fullsized":"2024-01-16T15:00:05.127755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:57:14.928029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.616725+0000","last_change":"2024-01-16T15:00:02.634166+0000","last_active":"2024-01-16T15:00:03.616725+0000","last_peered":"2024-01-16T15:00:03.616725+0000","last_clean":"2024-01-16T15:00:03.616725+0000","last_became_active":"2024-01-16T15:00:02.633874+0000","last_became_peered":"2024-01-16T15:00:02.633874+0000","last_unstale":"2024-01-16T15:00:03.616725+0000","last_undegraded":"2024-01-16T15:00:03.616725+0000","last_fullsized":"2024-01-16T15:00:03.616725+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:39:13.413754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1719999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2809999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.419}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.181}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5649999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7350000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6779999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4450000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9750000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1619999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2190000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6180000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7230000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.897}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.99}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.093}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.149}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2090000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.528}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7470000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0289999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.968}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0670000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0449999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.179}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3380000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.873}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5489999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1419999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1719999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95299999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1910000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":806912,"data_stored":660897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.135}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2949999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2709999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.323}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-16T15:00:08.448 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-16T15:00:08.449 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-01-16T15:00:08.449 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-01-16T15:00:08.449 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-01-16T15:00:08.449 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-01-16T15:00:08.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.711+0000 7f6a38a00640 1 Processor -- start 2024-01-16T15:00:08.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.711+0000 7f6a38a00640 1 -- start start 2024-01-16T15:00:08.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.713+0000 7f6a38a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6a340703f0 0x7f6a34162970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:08.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.713+0000 7f6a38a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6a340cd160 0x7f6a340cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:08.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.713+0000 7f6a38a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a34162eb0 con 0x7f6a3414b100 2024-01-16T15:00:08.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.713+0000 7f6a38a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a340cdb10 con 0x7f6a340703f0 2024-01-16T15:00:08.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.713+0000 7f6a38a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a341654a0 con 0x7f6a340cd160 2024-01-16T15:00:08.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.713+0000 7f6a29c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6a340703f0 0x7f6a34162970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:08.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.713+0000 7f6a29c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6a340703f0 0x7f6a34162970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42430/0 (socket says 172.21.15.143:42430) 2024-01-16T15:00:08.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.713+0000 7f6a29c00640 1 -- 172.21.15.143:0/3986227423 learned_addr learned my addr 172.21.15.143:0/3986227423 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:08.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.714+0000 7f6a33200640 1 --2- 172.21.15.143:0/3986227423 >> v2:172.21.15.204:3300/0 conn(0x7f6a340cd160 0x7f6a340cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:08.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.714+0000 7f6a29c00640 1 -- 172.21.15.143:0/3986227423 >> v2:172.21.15.204:3300/0 conn(0x7f6a340cd160 msgr2=0x7f6a340cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:08.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.714+0000 7f6a29c00640 1 --2- 172.21.15.143:0/3986227423 >> v2:172.21.15.204:3300/0 conn(0x7f6a340cd160 0x7f6a340cd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:08.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.714+0000 7f6a29c00640 1 -- 172.21.15.143:0/3986227423 >> v1:172.21.15.204:6789/0 conn(0x7f6a3414b100 legacy=0x7f6a34165c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:08.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.714+0000 7f6a29c00640 1 -- 172.21.15.143:0/3986227423 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a340ccf00 con 0x7f6a340703f0 2024-01-16T15:00:08.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.715+0000 7f6a29c00640 1 --2- 172.21.15.143:0/3986227423 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a340703f0 0x7f6a34162970 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6a340aea00 tx=0x7f6a1c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4285ade322a0b773 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:08.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.716+0000 7f6a31e00640 1 -- 172.21.15.143:0/3986227423 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c0317e0 con 0x7f6a340703f0 2024-01-16T15:00:08.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.716+0000 7f6a31e00640 1 -- 172.21.15.143:0/3986227423 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a1c031940 con 0x7f6a340703f0 2024-01-16T15:00:08.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.716+0000 7f6a31e00640 1 -- 172.21.15.143:0/3986227423 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c031cd0 con 0x7f6a340703f0 2024-01-16T15:00:08.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.716+0000 7f6a38a00640 1 -- 172.21.15.143:0/3986227423 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a340703f0 msgr2=0x7f6a34162970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:08.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.716+0000 7f6a38a00640 1 --2- 172.21.15.143:0/3986227423 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a340703f0 0x7f6a34162970 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6a340aea00 tx=0x7f6a1c02b880 comp rx=0 tx=0).stop 2024-01-16T15:00:08.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.716+0000 7f6a38a00640 1 -- 172.21.15.143:0/3986227423 shutdown_connections 2024-01-16T15:00:08.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.716+0000 7f6a38a00640 1 --2- 172.21.15.143:0/3986227423 >> v2:172.21.15.204:3300/0 conn(0x7f6a340cd160 0x7f6a340cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:08.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.716+0000 7f6a38a00640 1 --2- 172.21.15.143:0/3986227423 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a340703f0 0x7f6a34162970 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:08.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.716+0000 7f6a38a00640 1 -- 172.21.15.143:0/3986227423 >> 172.21.15.143:0/3986227423 conn(0x7f6a340cb830 msgr2=0x7f6a340cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:08.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.716+0000 7f6a38a00640 1 -- 172.21.15.143:0/3986227423 shutdown_connections 2024-01-16T15:00:08.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.716+0000 7f6a38a00640 1 -- 172.21.15.143:0/3986227423 wait complete. 2024-01-16T15:00:08.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.718+0000 7f6a38a00640 1 Processor -- start 2024-01-16T15:00:08.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.718+0000 7f6a38a00640 1 -- start start 2024-01-16T15:00:08.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.719+0000 7f6a38a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a340703f0 0x7f6a34146f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:08.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.719+0000 7f6a38a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a340cd160 0x7f6a340b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:08.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.719+0000 7f6a38a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a3414b100 0x7f6a34164630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:08.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.719+0000 7f6a38a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a340a9510 con 0x7f6a3414b100 2024-01-16T15:00:08.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.719+0000 7f6a38a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a34147490 con 0x7f6a340cd160 2024-01-16T15:00:08.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.719+0000 7f6a38a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a340ac2b0 con 0x7f6a340703f0 2024-01-16T15:00:08.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.719+0000 7f6a33200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a3414b100 0x7f6a34164630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:08.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.719+0000 7f6a33200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a3414b100 0x7f6a34164630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42440/0 (socket says 172.21.15.143:42440) 2024-01-16T15:00:08.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.719+0000 7f6a33200640 1 -- 172.21.15.143:0/3790222902 learned_addr learned my addr 172.21.15.143:0/3790222902 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:08.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.720+0000 7f6a29c00640 1 --2- 172.21.15.143:0/3790222902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a340cd160 0x7f6a340b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:08.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.720+0000 7f6a32800640 1 --2- 172.21.15.143:0/3790222902 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a340703f0 0x7f6a34146f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:08.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.720+0000 7f6a33200640 1 -- 172.21.15.143:0/3790222902 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a340703f0 msgr2=0x7f6a34146f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:08.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.720+0000 7f6a33200640 1 --2- 172.21.15.143:0/3790222902 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a340703f0 0x7f6a34146f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:08.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.720+0000 7f6a33200640 1 -- 172.21.15.143:0/3790222902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a340cd160 msgr2=0x7f6a340b4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:08.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.720+0000 7f6a33200640 1 --2- 172.21.15.143:0/3790222902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a340cd160 0x7f6a340b4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:08.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.720+0000 7f6a33200640 1 -- 172.21.15.143:0/3790222902 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a1c0094d0 con 0x7f6a3414b100 2024-01-16T15:00:08.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.720+0000 7f6a29c00640 1 --2- 172.21.15.143:0/3790222902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a340cd160 0x7f6a340b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:08.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.720+0000 7f6a33200640 1 --2- 172.21.15.143:0/3790222902 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a3414b100 0x7f6a34164630 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c007e40 tx=0x7f6a2c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:08.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.720+0000 7f6a2b400640 1 -- 172.21.15.143:0/3790222902 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c014620 con 0x7f6a3414b100 2024-01-16T15:00:08.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.720+0000 7f6a2b400640 1 -- 172.21.15.143:0/3790222902 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2c0147a0 con 0x7f6a3414b100 2024-01-16T15:00:08.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.720+0000 7f6a38a00640 1 -- 172.21.15.143:0/3790222902 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a340a89f0 con 0x7f6a3414b100 2024-01-16T15:00:08.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.720+0000 7f6a2b400640 1 -- 172.21.15.143:0/3790222902 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c014b50 con 0x7f6a3414b100 2024-01-16T15:00:08.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.721+0000 7f6a38a00640 1 -- 172.21.15.143:0/3790222902 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a340a9230 con 0x7f6a3414b100 2024-01-16T15:00:08.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.722+0000 7f6a38a00640 1 -- 172.21.15.143:0/3790222902 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a3406dae0 con 0x7f6a3414b100 2024-01-16T15:00:08.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.722+0000 7f6a2b400640 1 -- 172.21.15.143:0/3790222902 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f6a2c014cb0 con 0x7f6a3414b100 2024-01-16T15:00:08.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.723+0000 7f6a2b400640 1 --2- 172.21.15.143:0/3790222902 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6a0c06b2c0 0x7f6a0c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:08.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.723+0000 7f6a2b400640 1 -- 172.21.15.143:0/3790222902 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a2c08a540 con 0x7f6a3414b100 2024-01-16T15:00:08.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.724+0000 7f6a32800640 1 --2- 172.21.15.143:0/3790222902 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6a0c06b2c0 0x7f6a0c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:08.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.724+0000 7f6a32800640 1 --2- 172.21.15.143:0/3790222902 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6a0c06b2c0 0x7f6a0c06d780 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6a24006fd0 tx=0x7f6a24008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:08.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.728+0000 7f6a2b400640 1 -- 172.21.15.143:0/3790222902 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6a2c0553a0 con 0x7f6a3414b100 2024-01-16T15:00:08.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.965+0000 7f6a38a00640 1 -- 172.21.15.143:0/3790222902 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f6a340ce5b0 con 0x7f6a3414b100 2024-01-16T15:00:08.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.968+0000 7f6a2b400640 1 -- 172.21.15.143:0/3790222902 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171565 (secure 0 0 0) 0x7f6a2c0589c0 con 0x7f6a3414b100 2024-01-16T15:00:08.951 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:00:08.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.972+0000 7f6a38a00640 1 -- 172.21.15.143:0/3790222902 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6a0c06b2c0 msgr2=0x7f6a0c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:08.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.972+0000 7f6a38a00640 1 --2- 172.21.15.143:0/3790222902 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6a0c06b2c0 0x7f6a0c06d780 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6a24006fd0 tx=0x7f6a24008040 comp rx=0 tx=0).stop 2024-01-16T15:00:08.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.972+0000 7f6a38a00640 1 -- 172.21.15.143:0/3790222902 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a3414b100 msgr2=0x7f6a34164630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:08.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.972+0000 7f6a38a00640 1 --2- 172.21.15.143:0/3790222902 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a3414b100 0x7f6a34164630 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c007e40 tx=0x7f6a2c00e4b0 comp rx=0 tx=0).stop 2024-01-16T15:00:08.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.973+0000 7f6a38a00640 1 -- 172.21.15.143:0/3790222902 shutdown_connections 2024-01-16T15:00:08.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.973+0000 7f6a38a00640 1 --2- 172.21.15.143:0/3790222902 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6a0c06b2c0 0x7f6a0c06d780 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:08.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.973+0000 7f6a38a00640 1 --2- 172.21.15.143:0/3790222902 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a3414b100 0x7f6a34164630 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:08.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.973+0000 7f6a38a00640 1 --2- 172.21.15.143:0/3790222902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a340cd160 0x7f6a340b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:08.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.973+0000 7f6a38a00640 1 --2- 172.21.15.143:0/3790222902 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a340703f0 0x7f6a34146f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:08.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.973+0000 7f6a38a00640 1 -- 172.21.15.143:0/3790222902 >> 172.21.15.143:0/3790222902 conn(0x7f6a340cb830 msgr2=0x7f6a340cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:08.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.973+0000 7f6a38a00640 1 -- 172.21.15.143:0/3790222902 shutdown_connections 2024-01-16T15:00:08.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:08.973+0000 7f6a38a00640 1 -- 172.21.15.143:0/3790222902 wait complete. 2024-01-16T15:00:08.973 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":5,"active_gid":4113,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3153492115},{"type":"v1","addr":"172.21.15.143:6833","nonce":3153492115}]},"active_addr":"172.21.15.143:6833/3153492115","active_change":"2024-01-16T14:59:43.635899+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4109,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"clone_delay_every_regfile":{"name":"clone_delay_every_regfile","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Sleep for specified seconds after copying a regfile","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"clone_delay_every_regfile":{"name":"clone_delay_every_regfile","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Sleep for specified seconds after copying a regfile","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.143:0","nonce":1001382857}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.143:0","nonce":1538912492}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.143:0","nonce":3523023117}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.143:0","nonce":1043381577}]}]} 2024-01-16T15:00:08.976 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-01-16T15:00:08.976 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-01-16T15:00:08.977 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T15:00:09.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.206+0000 7f8a96600640 1 Processor -- start 2024-01-16T15:00:09.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.207+0000 7f8a96600640 1 -- start start 2024-01-16T15:00:09.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.208+0000 7f8a96600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8a9015df80 0x7f8a900ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:09.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.208+0000 7f8a96600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a900cd470 0x7f8a9015b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:09.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.208+0000 7f8a96600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a9015e3f0 con 0x7f8a900b2250 2024-01-16T15:00:09.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.208+0000 7f8a96600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a9015a990 con 0x7f8a9015df80 2024-01-16T15:00:09.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.208+0000 7f8a96600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9015ab80 con 0x7f8a900cd470 2024-01-16T15:00:09.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.208+0000 7f8a8ea00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8a9015df80 0x7f8a900ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:09.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.208+0000 7f8a8f400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8a900b2250 0x7f8a900b2630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54092/0 (socket says 172.21.15.143:54092) 2024-01-16T15:00:09.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.208+0000 7f8a8f400640 1 -- 172.21.15.143:0/3874612003 learned_addr learned my addr 172.21.15.143:0/3874612003 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:09.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.209+0000 7f8a8fe00640 1 --2- 172.21.15.143:0/3874612003 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a900cd470 0x7f8a9015b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:09.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.209+0000 7f8a8ea00640 1 -- 172.21.15.143:0/3874612003 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a900cd470 msgr2=0x7f8a9015b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:09.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.209+0000 7f8a8ea00640 1 --2- 172.21.15.143:0/3874612003 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a900cd470 0x7f8a9015b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:09.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.209+0000 7f8a8ea00640 1 -- 172.21.15.143:0/3874612003 >> v1:172.21.15.143:6789/0 conn(0x7f8a900b2250 legacy=0x7f8a900b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:09.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.209+0000 7f8a8ea00640 1 -- 172.21.15.143:0/3874612003 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a900b2d40 con 0x7f8a9015df80 2024-01-16T15:00:09.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.210+0000 7f8a8ea00640 1 --2- 172.21.15.143:0/3874612003 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a9015df80 0x7f8a900ccf30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8a780097a0 tx=0x7f8a7802dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892cac0a82f572c9 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:09.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.210+0000 7f8a8e000640 1 -- 172.21.15.143:0/3874612003 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7802f880 con 0x7f8a9015df80 2024-01-16T15:00:09.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.210+0000 7f8a8e000640 1 -- 172.21.15.143:0/3874612003 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7802f9e0 con 0x7f8a9015df80 2024-01-16T15:00:09.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.210+0000 7f8a8e000640 1 -- 172.21.15.143:0/3874612003 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7802fd70 con 0x7f8a9015df80 2024-01-16T15:00:09.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.211+0000 7f8a96600640 1 -- 172.21.15.143:0/3874612003 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a9015df80 msgr2=0x7f8a900ccf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:09.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.211+0000 7f8a96600640 1 --2- 172.21.15.143:0/3874612003 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a9015df80 0x7f8a900ccf30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8a780097a0 tx=0x7f8a7802dac0 comp rx=0 tx=0).stop 2024-01-16T15:00:09.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.211+0000 7f8a96600640 1 -- 172.21.15.143:0/3874612003 shutdown_connections 2024-01-16T15:00:09.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.211+0000 7f8a96600640 1 --2- 172.21.15.143:0/3874612003 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a900cd470 0x7f8a9015b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:09.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.211+0000 7f8a96600640 1 --2- 172.21.15.143:0/3874612003 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a9015df80 0x7f8a900ccf30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:09.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.211+0000 7f8a96600640 1 -- 172.21.15.143:0/3874612003 >> 172.21.15.143:0/3874612003 conn(0x7f8a9006a920 msgr2=0x7f8a9006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:09.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.211+0000 7f8a96600640 1 -- 172.21.15.143:0/3874612003 shutdown_connections 2024-01-16T15:00:09.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.211+0000 7f8a96600640 1 -- 172.21.15.143:0/3874612003 wait complete. 2024-01-16T15:00:09.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.214+0000 7f8a96600640 1 Processor -- start 2024-01-16T15:00:09.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.214+0000 7f8a96600640 1 -- start start 2024-01-16T15:00:09.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.215+0000 7f8a96600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a900b2250 0x7f8a90071880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:09.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.215+0000 7f8a96600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a900cd470 0x7f8a900adc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:09.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.215+0000 7f8a96600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a9015df80 0x7f8a900ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:09.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.215+0000 7f8a96600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a90155100 con 0x7f8a900cd470 2024-01-16T15:00:09.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.215+0000 7f8a96600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a90155290 con 0x7f8a9015df80 2024-01-16T15:00:09.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.215+0000 7f8a96600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a900c90a0 con 0x7f8a900b2250 2024-01-16T15:00:09.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.215+0000 7f8a8ea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a900cd470 0x7f8a900adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:09.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.215+0000 7f8a8ea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a900cd470 0x7f8a900adc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42458/0 (socket says 172.21.15.143:42458) 2024-01-16T15:00:09.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.215+0000 7f8a8ea00640 1 -- 172.21.15.143:0/3760829211 learned_addr learned my addr 172.21.15.143:0/3760829211 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:09.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.216+0000 7f8a8f400640 1 --2- 172.21.15.143:0/3760829211 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a900b2250 0x7f8a90071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:09.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.216+0000 7f8a8ea00640 1 -- 172.21.15.143:0/3760829211 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a900b2250 msgr2=0x7f8a90071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:09.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.216+0000 7f8a8ea00640 1 --2- 172.21.15.143:0/3760829211 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a900b2250 0x7f8a90071880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:09.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.216+0000 7f8a8fe00640 1 --2- 172.21.15.143:0/3760829211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a9015df80 0x7f8a900ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:09.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.216+0000 7f8a8ea00640 1 -- 172.21.15.143:0/3760829211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a9015df80 msgr2=0x7f8a900ae1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:09.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.216+0000 7f8a8ea00640 1 --2- 172.21.15.143:0/3760829211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a9015df80 0x7f8a900ae1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:09.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.216+0000 7f8a8ea00640 1 -- 172.21.15.143:0/3760829211 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a78009400 con 0x7f8a900cd470 2024-01-16T15:00:09.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.216+0000 7f8a8ea00640 1 --2- 172.21.15.143:0/3760829211 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a900cd470 0x7f8a900adc90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8a7802e0d0 tx=0x7f8a780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:09.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.216+0000 7f8a87400640 1 -- 172.21.15.143:0/3760829211 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a78030a90 con 0x7f8a900cd470 2024-01-16T15:00:09.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.216+0000 7f8a87400640 1 -- 172.21.15.143:0/3760829211 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a78030c10 con 0x7f8a900cd470 2024-01-16T15:00:09.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.216+0000 7f8a96600640 1 -- 172.21.15.143:0/3760829211 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a900c92f0 con 0x7f8a900cd470 2024-01-16T15:00:09.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.216+0000 7f8a87400640 1 -- 172.21.15.143:0/3760829211 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a78042410 con 0x7f8a900cd470 2024-01-16T15:00:09.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.217+0000 7f8a96600640 1 -- 172.21.15.143:0/3760829211 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a900c9680 con 0x7f8a900cd470 2024-01-16T15:00:09.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.218+0000 7f8a96600640 1 -- 172.21.15.143:0/3760829211 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a90155400 con 0x7f8a900cd470 2024-01-16T15:00:09.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.219+0000 7f8a87400640 1 -- 172.21.15.143:0/3760829211 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f8a78039400 con 0x7f8a900cd470 2024-01-16T15:00:09.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.220+0000 7f8a87400640 1 --2- 172.21.15.143:0/3760829211 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f8a6806b2f0 0x7f8a6806d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:09.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.220+0000 7f8a87400640 1 -- 172.21.15.143:0/3760829211 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a780aba90 con 0x7f8a900cd470 2024-01-16T15:00:09.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.220+0000 7f8a8f400640 1 --2- 172.21.15.143:0/3760829211 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f8a6806b2f0 0x7f8a6806d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:09.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.221+0000 7f8a8f400640 1 --2- 172.21.15.143:0/3760829211 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f8a6806b2f0 0x7f8a6806d7b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8a80005c00 tx=0x7f8a80009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:09.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.225+0000 7f8a87400640 1 -- 172.21.15.143:0/3760829211 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8a90155400 con 0x7f8a900cd470 2024-01-16T15:00:09.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.415+0000 7f8a96600640 1 -- 172.21.15.143:0/3760829211 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a90155400 con 0x7f8a900cd470 2024-01-16T15:00:09.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.418+0000 7f8a87400640 1 -- 172.21.15.143:0/3760829211 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16565 (secure 0 0 0) 0x7f8a90155400 con 0x7f8a900cd470 2024-01-16T15:00:09.398 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:00:09.398 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0","created":"2024-01-16T14:59:38.532601+0000","modified":"2024-01-16T15:00:04.543620+0000","last_up_change":"2024-01-16T14:59:50.983699+0000","last_in_change":"2024-01-16T14:59:42.056297+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:59:51.727720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T15:00:01.010472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T15:00:02.041228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f4c8891d-ab29-472a-907d-8446f20f7012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6801","nonce":1569293626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6805","nonce":1569293626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6814","nonce":1569293626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6809","nonce":1569293626}]},"public_addr":"172.21.15.143:6801/1569293626","cluster_addr":"172.21.15.143:6805/1569293626","heartbeat_back_addr":"172.21.15.143:6814/1569293626","heartbeat_front_addr":"172.21.15.143:6809/1569293626","state":["exists","up"]},{"osd":1,"uuid":"c4a2788d-8b89-4992-82b2-7f4b9debd509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6804","nonce":1276841168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6808","nonce":1276841168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6813","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6815","nonce":1276841168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6811","nonce":1276841168}]},"public_addr":"172.21.15.143:6804/1276841168","cluster_addr":"172.21.15.143:6808/1276841168","heartbeat_back_addr":"172.21.15.143:6815/1276841168","heartbeat_front_addr":"172.21.15.143:6811/1276841168","state":["exists","up"]},{"osd":2,"uuid":"64645c1d-31b2-4794-b0f2-7ccf8df30393","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6817","nonce":1217670170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6819","nonce":1217670170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6823","nonce":1217670170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6821","nonce":1217670170}]},"public_addr":"172.21.15.143:6817/1217670170","cluster_addr":"172.21.15.143:6819/1217670170","heartbeat_back_addr":"172.21.15.143:6823/1217670170","heartbeat_front_addr":"172.21.15.143:6821/1217670170","state":["exists","up"]},{"osd":3,"uuid":"73674129-a84e-4d31-8247-7d0179ae6df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6825","nonce":3414188012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6827","nonce":3414188012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6831","nonce":3414188012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6829","nonce":3414188012}]},"public_addr":"172.21.15.143:6825/3414188012","cluster_addr":"172.21.15.143:6827/3414188012","heartbeat_back_addr":"172.21.15.143:6831/3414188012","heartbeat_front_addr":"172.21.15.143:6829/3414188012","state":["exists","up"]},{"osd":4,"uuid":"cea28c9b-2f1c-4087-b415-9960789131d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6811","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6812","nonce":1070255716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6816","nonce":1070255716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6823","nonce":1070255716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6820","nonce":1070255716}]},"public_addr":"172.21.15.204:6812/1070255716","cluster_addr":"172.21.15.204:6816/1070255716","heartbeat_back_addr":"172.21.15.204:6823/1070255716","heartbeat_front_addr":"172.21.15.204:6820/1070255716","state":["exists","up"]},{"osd":5,"uuid":"33b63560-5574-4e78-983f-b398d67a2cb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6809","nonce":3605417837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6813","nonce":3605417837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6819","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6821","nonce":3605417837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6817","nonce":3605417837}]},"public_addr":"172.21.15.204:6809/3605417837","cluster_addr":"172.21.15.204:6813/3605417837","heartbeat_back_addr":"172.21.15.204:6821/3605417837","heartbeat_front_addr":"172.21.15.204:6817/3605417837","state":["exists","up"]},{"osd":6,"uuid":"df397427-8940-4eee-9c79-eece669d735d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6801","nonce":1085000534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6803","nonce":1085000534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6808","nonce":1085000534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6805","nonce":1085000534}]},"public_addr":"172.21.15.204:6801/1085000534","cluster_addr":"172.21.15.204:6803/1085000534","heartbeat_back_addr":"172.21.15.204:6808/1085000534","heartbeat_front_addr":"172.21.15.204:6805/1085000534","state":["exists","up"]},{"osd":7,"uuid":"a2e5a15c-8312-405a-ac02-f7e1fc9df2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6825","nonce":1337220609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6827","nonce":1337220609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6831","nonce":1337220609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6829","nonce":1337220609}]},"public_addr":"172.21.15.204:6825/1337220609","cluster_addr":"172.21.15.204:6827/1337220609","heartbeat_back_addr":"172.21.15.204:6831/1337220609","heartbeat_front_addr":"172.21.15.204:6829/1337220609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:49.984507+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.395306+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.632998+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.373627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.342920+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.438366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.331926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.406657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T15:00:09.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.420+0000 7f8a96600640 1 -- 172.21.15.143:0/3760829211 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f8a6806b2f0 msgr2=0x7f8a6806d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:09.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.420+0000 7f8a96600640 1 --2- 172.21.15.143:0/3760829211 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f8a6806b2f0 0x7f8a6806d7b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8a80005c00 tx=0x7f8a80009040 comp rx=0 tx=0).stop 2024-01-16T15:00:09.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.420+0000 7f8a96600640 1 -- 172.21.15.143:0/3760829211 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a900cd470 msgr2=0x7f8a900adc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:09.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.420+0000 7f8a96600640 1 --2- 172.21.15.143:0/3760829211 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a900cd470 0x7f8a900adc90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8a7802e0d0 tx=0x7f8a780046c0 comp rx=0 tx=0).stop 2024-01-16T15:00:09.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.420+0000 7f8a96600640 1 -- 172.21.15.143:0/3760829211 shutdown_connections 2024-01-16T15:00:09.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.420+0000 7f8a96600640 1 --2- 172.21.15.143:0/3760829211 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f8a6806b2f0 0x7f8a6806d7b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:09.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.420+0000 7f8a96600640 1 --2- 172.21.15.143:0/3760829211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a9015df80 0x7f8a900ae1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:09.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.420+0000 7f8a96600640 1 --2- 172.21.15.143:0/3760829211 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a900cd470 0x7f8a900adc90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:09.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.420+0000 7f8a96600640 1 --2- 172.21.15.143:0/3760829211 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a900b2250 0x7f8a90071880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:09.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.420+0000 7f8a96600640 1 -- 172.21.15.143:0/3760829211 >> 172.21.15.143:0/3760829211 conn(0x7f8a9006a920 msgr2=0x7f8a900b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:09.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.421+0000 7f8a96600640 1 -- 172.21.15.143:0/3760829211 shutdown_connections 2024-01-16T15:00:09.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.421+0000 7f8a96600640 1 -- 172.21.15.143:0/3760829211 wait complete. 2024-01-16T15:00:09.420 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-01-16T15:00:09.420 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T15:00:09.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.651+0000 7f1924a00640 1 Processor -- start 2024-01-16T15:00:09.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.651+0000 7f1924a00640 1 -- start start 2024-01-16T15:00:09.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.653+0000 7f1924a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f191c15c7f0 0x7f191c06db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:09.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.653+0000 7f1924a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f191c0cf450 0x7f191c0cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:09.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.653+0000 7f1924a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f191c0cdfb0 0x7f191c0ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:09.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.653+0000 7f1924a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f191c156010 con 0x7f191c0cdfb0 2024-01-16T15:00:09.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.653+0000 7f1924a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f191c06ce30 con 0x7f191c0cf450 2024-01-16T15:00:09.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.653+0000 7f1924a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f191c0cfe10 con 0x7f191c15c7f0 2024-01-16T15:00:09.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.653+0000 7f1922a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f191c0cdfb0 0x7f191c0ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:09.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.653+0000 7f1922a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f191c0cdfb0 0x7f191c0ce440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42464/0 (socket says 172.21.15.143:42464) 2024-01-16T15:00:09.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.653+0000 7f1922a00640 1 -- 172.21.15.143:0/3136971185 learned_addr learned my addr 172.21.15.143:0/3136971185 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:09.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.653+0000 7f1922000640 1 --2- 172.21.15.143:0/3136971185 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f191c15c7f0 0x7f191c06db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:09.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.653+0000 7f1921600640 1 --2- 172.21.15.143:0/3136971185 >> v2:172.21.15.204:3300/0 conn(0x7f191c0cf450 0x7f191c0cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:09.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.653+0000 7f1922a00640 1 -- 172.21.15.143:0/3136971185 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f191c15c7f0 msgr2=0x7f191c06db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:09.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.654+0000 7f1922a00640 1 --2- 172.21.15.143:0/3136971185 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f191c15c7f0 0x7f191c06db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:09.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.654+0000 7f1922a00640 1 -- 172.21.15.143:0/3136971185 >> v2:172.21.15.204:3300/0 conn(0x7f191c0cf450 msgr2=0x7f191c0cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:09.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.654+0000 7f1922a00640 1 --2- 172.21.15.143:0/3136971185 >> v2:172.21.15.204:3300/0 conn(0x7f191c0cf450 0x7f191c0cf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:09.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.654+0000 7f1922a00640 1 -- 172.21.15.143:0/3136971185 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f191c0cece0 con 0x7f191c0cdfb0 2024-01-16T15:00:09.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.654+0000 7f1921600640 1 --2- 172.21.15.143:0/3136971185 >> v2:172.21.15.204:3300/0 conn(0x7f191c0cf450 0x7f191c0cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:09.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.655+0000 7f1922a00640 1 --2- 172.21.15.143:0/3136971185 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f191c0cdfb0 0x7f191c0ce440 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f191c06b550 tx=0x7f191402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b551e785d32945ea server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:09.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.657+0000 7f1920c00640 1 -- 172.21.15.143:0/3136971185 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191402ca80 con 0x7f191c0cdfb0 2024-01-16T15:00:09.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.657+0000 7f1920c00640 1 -- 172.21.15.143:0/3136971185 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f191402cbe0 con 0x7f191c0cdfb0 2024-01-16T15:00:09.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.657+0000 7f1920c00640 1 -- 172.21.15.143:0/3136971185 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19140343f0 con 0x7f191c0cdfb0 2024-01-16T15:00:09.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.657+0000 7f1924a00640 1 -- 172.21.15.143:0/3136971185 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f191c0cdfb0 msgr2=0x7f191c0ce440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:09.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.657+0000 7f1924a00640 1 --2- 172.21.15.143:0/3136971185 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f191c0cdfb0 0x7f191c0ce440 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f191c06b550 tx=0x7f191402bee0 comp rx=0 tx=0).stop 2024-01-16T15:00:09.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.658+0000 7f1924a00640 1 -- 172.21.15.143:0/3136971185 shutdown_connections 2024-01-16T15:00:09.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.658+0000 7f1924a00640 1 --2- 172.21.15.143:0/3136971185 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f191c0cdfb0 0x7f191c0ce440 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:09.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.658+0000 7f1924a00640 1 --2- 172.21.15.143:0/3136971185 >> v2:172.21.15.204:3300/0 conn(0x7f191c0cf450 0x7f191c0cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:09.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.658+0000 7f1924a00640 1 --2- 172.21.15.143:0/3136971185 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f191c15c7f0 0x7f191c06db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:09.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.658+0000 7f1924a00640 1 -- 172.21.15.143:0/3136971185 >> 172.21.15.143:0/3136971185 conn(0x7f191c06a920 msgr2=0x7f191c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:09.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.658+0000 7f1924a00640 1 -- 172.21.15.143:0/3136971185 shutdown_connections 2024-01-16T15:00:09.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.658+0000 7f1924a00640 1 -- 172.21.15.143:0/3136971185 wait complete. 2024-01-16T15:00:09.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.660+0000 7f1924a00640 1 Processor -- start 2024-01-16T15:00:09.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.660+0000 7f1924a00640 1 -- start start 2024-01-16T15:00:09.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.661+0000 7f1924a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f191c0cdfb0 0x7f191c0b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:09.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.661+0000 7f1924a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f191c0cf450 0x7f191c0ad010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:09.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.661+0000 7f1924a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f191c15c7f0 0x7f191c0ad550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:09.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.661+0000 7f1924a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191c0b3670 con 0x7f191c0cdfb0 2024-01-16T15:00:09.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.661+0000 7f1924a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191c151050 con 0x7f191c0cf450 2024-01-16T15:00:09.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.661+0000 7f1924a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f191c1575e0 con 0x7f191c15c7f0 2024-01-16T15:00:09.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.661+0000 7f1922000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f191c0cdfb0 0x7f191c0b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:09.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.661+0000 7f1922000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f191c0cdfb0 0x7f191c0b3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42470/0 (socket says 172.21.15.143:42470) 2024-01-16T15:00:09.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.661+0000 7f1922000640 1 -- 172.21.15.143:0/1340846607 learned_addr learned my addr 172.21.15.143:0/1340846607 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:09.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.662+0000 7f1922a00640 1 --2- 172.21.15.143:0/1340846607 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f191c15c7f0 0x7f191c0ad550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:09.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.662+0000 7f1921600640 1 --2- 172.21.15.143:0/1340846607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f191c0cf450 0x7f191c0ad010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:09.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.662+0000 7f1922000640 1 -- 172.21.15.143:0/1340846607 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f191c15c7f0 msgr2=0x7f191c0ad550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:09.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.662+0000 7f1922000640 1 --2- 172.21.15.143:0/1340846607 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f191c15c7f0 0x7f191c0ad550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:09.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.662+0000 7f1922000640 1 -- 172.21.15.143:0/1340846607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f191c0cf450 msgr2=0x7f191c0ad010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:09.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.662+0000 7f1922000640 1 --2- 172.21.15.143:0/1340846607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f191c0cf450 0x7f191c0ad010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:09.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.662+0000 7f1922000640 1 -- 172.21.15.143:0/1340846607 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1914009b30 con 0x7f191c0cdfb0 2024-01-16T15:00:09.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.662+0000 7f1922a00640 1 --2- 172.21.15.143:0/1340846607 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f191c15c7f0 0x7f191c0ad550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:09.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.662+0000 7f1922000640 1 --2- 172.21.15.143:0/1340846607 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f191c0cdfb0 0x7f191c0b3c50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f190c00d800 tx=0x7f190c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:09.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.662+0000 7f191a000640 1 -- 172.21.15.143:0/1340846607 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c00b6c0 con 0x7f191c0cdfb0 2024-01-16T15:00:09.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.662+0000 7f191a000640 1 -- 172.21.15.143:0/1340846607 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f190c00b820 con 0x7f191c0cdfb0 2024-01-16T15:00:09.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.662+0000 7f191a000640 1 -- 172.21.15.143:0/1340846607 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c00bbb0 con 0x7f191c0cdfb0 2024-01-16T15:00:09.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.662+0000 7f1924a00640 1 -- 172.21.15.143:0/1340846607 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f191c1572b0 con 0x7f191c0cdfb0 2024-01-16T15:00:09.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.662+0000 7f1924a00640 1 -- 172.21.15.143:0/1340846607 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f191c0b30b0 con 0x7f191c0cdfb0 2024-01-16T15:00:09.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.664+0000 7f1924a00640 1 -- 172.21.15.143:0/1340846607 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f191c067470 con 0x7f191c0cdfb0 2024-01-16T15:00:09.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.665+0000 7f191a000640 1 -- 172.21.15.143:0/1340846607 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f190c00bd10 con 0x7f191c0cdfb0 2024-01-16T15:00:09.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.665+0000 7f191a000640 1 --2- 172.21.15.143:0/1340846607 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f18f406b2f0 0x7f18f406d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:09.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.665+0000 7f191a000640 1 -- 172.21.15.143:0/1340846607 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f190c08a0e0 con 0x7f191c0cdfb0 2024-01-16T15:00:09.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.666+0000 7f1921600640 1 --2- 172.21.15.143:0/1340846607 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f18f406b2f0 0x7f18f406d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:09.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.666+0000 7f1921600640 1 --2- 172.21.15.143:0/1340846607 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f18f406b2f0 0x7f18f406d7b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f191c0addc0 tx=0x7f1904008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:09.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.670+0000 7f191a000640 1 -- 172.21.15.143:0/1340846607 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f190c054f50 con 0x7f191c0cdfb0 2024-01-16T15:00:09.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.861+0000 7f1924a00640 1 -- 172.21.15.143:0/1340846607 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f191c0cd730 con 0x7f191c0cdfb0 2024-01-16T15:00:09.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.863+0000 7f191a000640 1 -- 172.21.15.143:0/1340846607 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16565 (secure 0 0 0) 0x7f190c058570 con 0x7f191c0cdfb0 2024-01-16T15:00:09.843 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:00:09.844 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0","created":"2024-01-16T14:59:38.532601+0000","modified":"2024-01-16T15:00:04.543620+0000","last_up_change":"2024-01-16T14:59:50.983699+0000","last_in_change":"2024-01-16T14:59:42.056297+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:59:51.727720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T15:00:01.010472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T15:00:02.041228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f4c8891d-ab29-472a-907d-8446f20f7012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6801","nonce":1569293626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6805","nonce":1569293626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6814","nonce":1569293626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6809","nonce":1569293626}]},"public_addr":"172.21.15.143:6801/1569293626","cluster_addr":"172.21.15.143:6805/1569293626","heartbeat_back_addr":"172.21.15.143:6814/1569293626","heartbeat_front_addr":"172.21.15.143:6809/1569293626","state":["exists","up"]},{"osd":1,"uuid":"c4a2788d-8b89-4992-82b2-7f4b9debd509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6804","nonce":1276841168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6808","nonce":1276841168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6813","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6815","nonce":1276841168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6811","nonce":1276841168}]},"public_addr":"172.21.15.143:6804/1276841168","cluster_addr":"172.21.15.143:6808/1276841168","heartbeat_back_addr":"172.21.15.143:6815/1276841168","heartbeat_front_addr":"172.21.15.143:6811/1276841168","state":["exists","up"]},{"osd":2,"uuid":"64645c1d-31b2-4794-b0f2-7ccf8df30393","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6817","nonce":1217670170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6819","nonce":1217670170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6823","nonce":1217670170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6821","nonce":1217670170}]},"public_addr":"172.21.15.143:6817/1217670170","cluster_addr":"172.21.15.143:6819/1217670170","heartbeat_back_addr":"172.21.15.143:6823/1217670170","heartbeat_front_addr":"172.21.15.143:6821/1217670170","state":["exists","up"]},{"osd":3,"uuid":"73674129-a84e-4d31-8247-7d0179ae6df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6825","nonce":3414188012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6827","nonce":3414188012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6831","nonce":3414188012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6829","nonce":3414188012}]},"public_addr":"172.21.15.143:6825/3414188012","cluster_addr":"172.21.15.143:6827/3414188012","heartbeat_back_addr":"172.21.15.143:6831/3414188012","heartbeat_front_addr":"172.21.15.143:6829/3414188012","state":["exists","up"]},{"osd":4,"uuid":"cea28c9b-2f1c-4087-b415-9960789131d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6811","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6812","nonce":1070255716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6816","nonce":1070255716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6823","nonce":1070255716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6820","nonce":1070255716}]},"public_addr":"172.21.15.204:6812/1070255716","cluster_addr":"172.21.15.204:6816/1070255716","heartbeat_back_addr":"172.21.15.204:6823/1070255716","heartbeat_front_addr":"172.21.15.204:6820/1070255716","state":["exists","up"]},{"osd":5,"uuid":"33b63560-5574-4e78-983f-b398d67a2cb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6809","nonce":3605417837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6813","nonce":3605417837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6819","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6821","nonce":3605417837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6817","nonce":3605417837}]},"public_addr":"172.21.15.204:6809/3605417837","cluster_addr":"172.21.15.204:6813/3605417837","heartbeat_back_addr":"172.21.15.204:6821/3605417837","heartbeat_front_addr":"172.21.15.204:6817/3605417837","state":["exists","up"]},{"osd":6,"uuid":"df397427-8940-4eee-9c79-eece669d735d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6801","nonce":1085000534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6803","nonce":1085000534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6808","nonce":1085000534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6805","nonce":1085000534}]},"public_addr":"172.21.15.204:6801/1085000534","cluster_addr":"172.21.15.204:6803/1085000534","heartbeat_back_addr":"172.21.15.204:6808/1085000534","heartbeat_front_addr":"172.21.15.204:6805/1085000534","state":["exists","up"]},{"osd":7,"uuid":"a2e5a15c-8312-405a-ac02-f7e1fc9df2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6825","nonce":1337220609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6827","nonce":1337220609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6831","nonce":1337220609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6829","nonce":1337220609}]},"public_addr":"172.21.15.204:6825/1337220609","cluster_addr":"172.21.15.204:6827/1337220609","heartbeat_back_addr":"172.21.15.204:6831/1337220609","heartbeat_front_addr":"172.21.15.204:6829/1337220609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:49.984507+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.395306+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.632998+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.373627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.342920+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.438366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.331926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.406657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T15:00:09.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.865+0000 7f1924a00640 1 -- 172.21.15.143:0/1340846607 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f18f406b2f0 msgr2=0x7f18f406d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:09.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.865+0000 7f1924a00640 1 --2- 172.21.15.143:0/1340846607 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f18f406b2f0 0x7f18f406d7b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f191c0addc0 tx=0x7f1904008040 comp rx=0 tx=0).stop 2024-01-16T15:00:09.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.865+0000 7f1924a00640 1 -- 172.21.15.143:0/1340846607 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f191c0cdfb0 msgr2=0x7f191c0b3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:09.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.865+0000 7f1924a00640 1 --2- 172.21.15.143:0/1340846607 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f191c0cdfb0 0x7f191c0b3c50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f190c00d800 tx=0x7f190c00dcc0 comp rx=0 tx=0).stop 2024-01-16T15:00:09.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.865+0000 7f1924a00640 1 -- 172.21.15.143:0/1340846607 shutdown_connections 2024-01-16T15:00:09.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.865+0000 7f1924a00640 1 --2- 172.21.15.143:0/1340846607 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f18f406b2f0 0x7f18f406d7b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:09.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.865+0000 7f1924a00640 1 --2- 172.21.15.143:0/1340846607 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f191c15c7f0 0x7f191c0ad550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:09.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.865+0000 7f1924a00640 1 --2- 172.21.15.143:0/1340846607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f191c0cf450 0x7f191c0ad010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:09.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.865+0000 7f1924a00640 1 --2- 172.21.15.143:0/1340846607 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f191c0cdfb0 0x7f191c0b3c50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:09.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.865+0000 7f1924a00640 1 -- 172.21.15.143:0/1340846607 >> 172.21.15.143:0/1340846607 conn(0x7f191c06a920 msgr2=0x7f191c0b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:09.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.866+0000 7f1924a00640 1 -- 172.21.15.143:0/1340846607 shutdown_connections 2024-01-16T15:00:09.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:09.866+0000 7f1924a00640 1 -- 172.21.15.143:0/1340846607 wait complete. 2024-01-16T15:00:09.865 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-01-16T15:00:09.866 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-01-16T15:00:09.866 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-01-16T15:00:09.867 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-01-16T15:00:09.867 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-01-16T15:00:09.867 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-01-16T15:00:09.868 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-01-16T15:00:09.868 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-01-16T15:00:10.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.189+0000 7ff1a7e00640 1 Processor -- start 2024-01-16T15:00:10.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.189+0000 7ff1a7e00640 1 -- start start 2024-01-16T15:00:10.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.190+0000 7f9a7e800640 1 Processor -- start 2024-01-16T15:00:10.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.190+0000 7f9a7e800640 1 -- start start 2024-01-16T15:00:10.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.191+0000 7ff1a7e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff1a0053c10 0x7ff1a0006ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.191+0000 7ff1a7e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1a0057d60 con 0x7ff1a00e76e0 2024-01-16T15:00:10.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.191+0000 7ff1a7e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1a0006230 con 0x7ff1a0058990 2024-01-16T15:00:10.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.191+0000 7ff1a7e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1a0006510 con 0x7ff1a0053c10 2024-01-16T15:00:10.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.191+0000 7fcc44800640 1 Processor -- start 2024-01-16T15:00:10.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.191+0000 7fcc44800640 1 -- start start 2024-01-16T15:00:10.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.191+0000 7ff1a6a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff1a00e76e0 0x7ff1a004ca00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54102/0 (socket says 172.21.15.143:54102) 2024-01-16T15:00:10.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.191+0000 7ff1a6a00640 1 -- 172.21.15.143:0/3711905198 learned_addr learned my addr 172.21.15.143:0/3711905198 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:10.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.192+0000 7ff1a7400640 1 --2- 172.21.15.143:0/3711905198 >> v2:172.21.15.143:3300/0 conn(0x7ff1a0053c10 0x7ff1a0006ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.192+0000 7f9a7e800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9a780d30c0 0x7f9a780d3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.193+0000 7f9a7e800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9a780b1190 0x7f9a780b15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.193+0000 7f9a7e800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a7815d8e0 con 0x7f9a7815d500 2024-01-16T15:00:10.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.193+0000 7f9a7e800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a780d14f0 con 0x7f9a780d30c0 2024-01-16T15:00:10.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.193+0000 7f9a7e800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a780afea0 con 0x7f9a780b1190 2024-01-16T15:00:10.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.193+0000 7f9a7ca00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9a780d30c0 0x7f9a780d3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.193+0000 7ff1a7400640 1 -- 172.21.15.143:0/3711905198 >> v1:172.21.15.204:6789/0 conn(0x7ff1a0058990 legacy=0x7ff1a0053500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.193+0000 7ff1a7400640 1 -- 172.21.15.143:0/3711905198 >> v1:172.21.15.143:6789/0 conn(0x7ff1a00e76e0 legacy=0x7ff1a004ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.193+0000 7ff1a7400640 1 -- 172.21.15.143:0/3711905198 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a00067f0 con 0x7ff1a0053c10 2024-01-16T15:00:10.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.193+0000 7fcc44800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcc340e3080 0x7fcc34057d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.194+0000 7fcc44800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fcc340542c0 0x7fcc34006890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.194+0000 7fcc44800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc34058290 con 0x7fcc3403e800 2024-01-16T15:00:10.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.194+0000 7fcc44800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc34006160 con 0x7fcc340e3080 2024-01-16T15:00:10.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.194+0000 7fcc44800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc34058530 con 0x7fcc340542c0 2024-01-16T15:00:10.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.194+0000 7fcc41e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcc340e3080 0x7fcc34057d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.194+0000 7fcc41e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcc340e3080 0x7fcc34057d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42498/0 (socket says 172.21.15.143:42498) 2024-01-16T15:00:10.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.194+0000 7fcc41e00640 1 -- 172.21.15.143:0/1310587413 learned_addr learned my addr 172.21.15.143:0/1310587413 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:10.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.195+0000 7fcc42800640 1 --2- 172.21.15.143:0/1310587413 >> v2:172.21.15.204:3300/0 conn(0x7fcc340542c0 0x7fcc34006890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.195+0000 7ff1a7400640 1 --2- 172.21.15.143:0/3711905198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1a0053c10 0x7ff1a0006ff0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8050140 tx=0x7ff1a80a62f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80074d993ed40818 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.195+0000 7fcc41e00640 1 -- 172.21.15.143:0/1310587413 >> v2:172.21.15.204:3300/0 conn(0x7fcc340542c0 msgr2=0x7fcc34006890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:10.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.195+0000 7fcc41e00640 1 --2- 172.21.15.143:0/1310587413 >> v2:172.21.15.204:3300/0 conn(0x7fcc340542c0 0x7fcc34006890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.195+0000 7fcc41e00640 1 -- 172.21.15.143:0/1310587413 >> v1:172.21.15.143:6789/0 conn(0x7fcc3403e800 legacy=0x7fcc34053bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:10.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.195+0000 7fcc41e00640 1 -- 172.21.15.143:0/1310587413 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc34058810 con 0x7fcc340e3080 2024-01-16T15:00:10.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.194+0000 7f9a7ca00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9a780d30c0 0x7f9a780d3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42492/0 (socket says 172.21.15.143:42492) 2024-01-16T15:00:10.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.195+0000 7f9a7ca00640 1 -- 172.21.15.143:0/2592264083 learned_addr learned my addr 172.21.15.143:0/2592264083 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:10.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.194+0000 7f9a7d400640 1 --1- 172.21.15.143:0/2592264083 >> v1:172.21.15.143:6789/0 conn(0x7f9a7815d500 0x7f9a780d0de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54112/0 (socket says 172.21.15.143:54112) 2024-01-16T15:00:10.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.196+0000 7ff1a5600640 1 -- 172.21.15.143:0/3711905198 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a80ae640 con 0x7ff1a0053c10 2024-01-16T15:00:10.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.194+0000 7f9a7de00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9a780b1190 0x7f9a780b15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.196+0000 7f9a77e00640 1 -- 172.21.15.143:0/2592264083 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2380361229 0 0) 0x7f9a60001110 con 0x7f9a7815d500 2024-01-16T15:00:10.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.196+0000 7f9a77e00640 1 -- 172.21.15.143:0/2592264083 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a58003520 con 0x7f9a7815d500 2024-01-16T15:00:10.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.196+0000 7ff1a5600640 1 -- 172.21.15.143:0/3711905198 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a80ae7e0 con 0x7ff1a0053c10 2024-01-16T15:00:10.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.196+0000 7ff1a5600640 1 -- 172.21.15.143:0/3711905198 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a80aec10 con 0x7ff1a0053c10 2024-01-16T15:00:10.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.197+0000 7ff1a7e00640 1 -- 172.21.15.143:0/3711905198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1a0053c10 msgr2=0x7ff1a0006ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.197+0000 7ff1a7e00640 1 --2- 172.21.15.143:0/3711905198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1a0053c10 0x7ff1a0006ff0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8050140 tx=0x7ff1a80a62f0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.196+0000 7fcc41e00640 1 --2- 172.21.15.143:0/1310587413 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc340e3080 0x7fcc34057d50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c0620a0 tx=0x7fcc3c0a63d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ef5adcd2af6d480 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.198+0000 7ff1a7e00640 1 -- 172.21.15.143:0/3711905198 shutdown_connections 2024-01-16T15:00:10.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.198+0000 7ff1a7e00640 1 --2- 172.21.15.143:0/3711905198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1a0053c10 0x7ff1a0006ff0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.198+0000 7ff1a7e00640 1 -- 172.21.15.143:0/3711905198 >> 172.21.15.143:0/3711905198 conn(0x7ff1a0058eb0 msgr2=0x7ff1a0059290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:10.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.198+0000 7fcc40a00640 1 -- 172.21.15.143:0/1310587413 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c0ab7d0 con 0x7fcc340e3080 2024-01-16T15:00:10.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.198+0000 7ff1a7e00640 1 -- 172.21.15.143:0/3711905198 shutdown_connections 2024-01-16T15:00:10.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.196+0000 7f9a7ca00640 1 -- 172.21.15.143:0/2592264083 >> v2:172.21.15.204:3300/0 conn(0x7f9a780b1190 msgr2=0x7f9a780b15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:10.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.196+0000 7f9a7ca00640 1 --2- 172.21.15.143:0/2592264083 >> v2:172.21.15.204:3300/0 conn(0x7f9a780b1190 0x7f9a780b15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.196+0000 7f9a7ca00640 1 -- 172.21.15.143:0/2592264083 >> v1:172.21.15.143:6789/0 conn(0x7f9a7815d500 legacy=0x7f9a780d0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.196+0000 7f9a7ca00640 1 -- 172.21.15.143:0/2592264083 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a780d3a90 con 0x7f9a780d30c0 2024-01-16T15:00:10.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.198+0000 7ff1a7e00640 1 -- 172.21.15.143:0/3711905198 wait complete. 2024-01-16T15:00:10.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.198+0000 7fcc40a00640 1 -- 172.21.15.143:0/1310587413 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc3c0a9050 con 0x7fcc340e3080 2024-01-16T15:00:10.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.198+0000 7fcc40a00640 1 -- 172.21.15.143:0/1310587413 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c0abc50 con 0x7fcc340e3080 2024-01-16T15:00:10.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.198+0000 7f9a7ca00640 1 --2- 172.21.15.143:0/2592264083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a780d30c0 0x7f9a780d3550 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9a7808d160 tx=0x7f9a6802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e6a0a5a0dfce9e server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.199+0000 7f9a77e00640 1 -- 172.21.15.143:0/2592264083 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a68030610 con 0x7f9a780d30c0 2024-01-16T15:00:10.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.199+0000 7f9a77e00640 1 -- 172.21.15.143:0/2592264083 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a68030770 con 0x7f9a780d30c0 2024-01-16T15:00:10.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.199+0000 7f9a77e00640 1 -- 172.21.15.143:0/2592264083 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a68030b40 con 0x7f9a780d30c0 2024-01-16T15:00:10.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.199+0000 7f9429e00640 1 Processor -- start 2024-01-16T15:00:10.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.199+0000 7fcc44800640 1 -- 172.21.15.143:0/1310587413 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc340e3080 msgr2=0x7fcc34057d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.199+0000 7fcc44800640 1 --2- 172.21.15.143:0/1310587413 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc340e3080 0x7fcc34057d50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c0620a0 tx=0x7fcc3c0a63d0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.199+0000 7f9429e00640 1 -- start start 2024-01-16T15:00:10.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.200+0000 7fcc44800640 1 -- 172.21.15.143:0/1310587413 shutdown_connections 2024-01-16T15:00:10.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.200+0000 7fcc44800640 1 --2- 172.21.15.143:0/1310587413 >> v2:172.21.15.204:3300/0 conn(0x7fcc340542c0 0x7fcc34006890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.200+0000 7fcc44800640 1 --2- 172.21.15.143:0/1310587413 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc340e3080 0x7fcc34057d50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.200+0000 7fcc44800640 1 -- 172.21.15.143:0/1310587413 >> 172.21.15.143:0/1310587413 conn(0x7fcc340559a0 msgr2=0x7fcc34055db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:10.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.200+0000 7f9a7e800640 1 -- 172.21.15.143:0/2592264083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a780d30c0 msgr2=0x7f9a780d3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.200+0000 7f9a7e800640 1 --2- 172.21.15.143:0/2592264083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a780d30c0 0x7f9a780d3550 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9a7808d160 tx=0x7f9a6802b960 comp rx=0 tx=0).stop 2024-01-16T15:00:10.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.200+0000 7fcc44800640 1 -- 172.21.15.143:0/1310587413 shutdown_connections 2024-01-16T15:00:10.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.200+0000 7fcc44800640 1 -- 172.21.15.143:0/1310587413 wait complete. 2024-01-16T15:00:10.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.200+0000 7f9a7e800640 1 -- 172.21.15.143:0/2592264083 shutdown_connections 2024-01-16T15:00:10.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.200+0000 7f9a7e800640 1 --2- 172.21.15.143:0/2592264083 >> v2:172.21.15.204:3300/0 conn(0x7f9a780b1190 0x7f9a780b15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.200+0000 7f9a7e800640 1 --2- 172.21.15.143:0/2592264083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a780d30c0 0x7f9a780d3550 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.200+0000 7f9a7e800640 1 -- 172.21.15.143:0/2592264083 >> 172.21.15.143:0/2592264083 conn(0x7f9a7808cc90 msgr2=0x7f9a7809c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:10.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.200+0000 7facf5800640 1 Processor -- start 2024-01-16T15:00:10.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.200+0000 7facf5800640 1 -- start start 2024-01-16T15:00:10.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.201+0000 7f9a7e800640 1 -- 172.21.15.143:0/2592264083 shutdown_connections 2024-01-16T15:00:10.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.201+0000 7f9429e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f94240b63d0 0x7f94240d3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.201+0000 7f9429e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94240ce270 0x7f94240d3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.201+0000 7f9429e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94240cea30 con 0x7f942415d490 2024-01-16T15:00:10.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.201+0000 7f9429e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94240d0450 con 0x7f94240b63d0 2024-01-16T15:00:10.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.201+0000 7f9429e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94240d0bf0 con 0x7f94240ce270 2024-01-16T15:00:10.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.202+0000 7f982d400640 1 Processor -- start 2024-01-16T15:00:10.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.202+0000 7f9428a00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f942415d490 0x7f94240cd8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:33110/0 (socket says 172.21.15.143:33110) 2024-01-16T15:00:10.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.202+0000 7f9428a00640 1 -- 172.21.15.143:0/1890630576 learned_addr learned my addr 172.21.15.143:0/1890630576 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:10.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.202+0000 7f9429400640 1 --2- 172.21.15.143:0/1890630576 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94240ce270 0x7f94240d3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.202+0000 7f9423e00640 1 --2- 172.21.15.143:0/1890630576 >> v2:172.21.15.204:3300/0 conn(0x7f94240b63d0 0x7f94240d3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.201+0000 7f9a7e800640 1 -- 172.21.15.143:0/2592264083 wait complete. 2024-01-16T15:00:10.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.203+0000 7fcc44800640 1 Processor -- start 2024-01-16T15:00:10.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.204+0000 7f9a7e800640 1 Processor -- start 2024-01-16T15:00:10.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.203+0000 7fcc44800640 1 -- start start 2024-01-16T15:00:10.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.203+0000 7ff1a7e00640 1 Processor -- start 2024-01-16T15:00:10.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.203+0000 7f9429400640 1 -- 172.21.15.143:0/1890630576 >> v2:172.21.15.204:3300/0 conn(0x7f94240b63d0 msgr2=0x7f94240d3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:10.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.203+0000 7f9429400640 1 --2- 172.21.15.143:0/1890630576 >> v2:172.21.15.204:3300/0 conn(0x7f94240b63d0 0x7f94240d3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.203+0000 7f9429400640 1 -- 172.21.15.143:0/1890630576 >> v1:172.21.15.204:6789/0 conn(0x7f942415d490 legacy=0x7f94240cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.203+0000 7f9429400640 1 -- 172.21.15.143:0/1890630576 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94240d2260 con 0x7f94240ce270 2024-01-16T15:00:10.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.203+0000 7f9423400640 1 -- 172.21.15.143:0/1890630576 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 83410173 0 0) 0x7f94240cea30 con 0x7f942415d490 2024-01-16T15:00:10.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.203+0000 7f1560a00640 1 Processor -- start 2024-01-16T15:00:10.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.204+0000 7ff1a7e00640 1 -- start start 2024-01-16T15:00:10.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.204+0000 7f9a7e800640 1 -- start start 2024-01-16T15:00:10.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.204+0000 7f1560a00640 1 -- start start 2024-01-16T15:00:10.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.202+0000 7f982d400640 1 -- start start 2024-01-16T15:00:10.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.204+0000 7facf5800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7facf00b1250 0x7facf00cd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.204+0000 7facf5800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7facf00ae310 0x7facf00cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.204+0000 7facf5800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facf0159a50 con 0x7facf0159670 2024-01-16T15:00:10.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.204+0000 7facf5800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facf015d6b0 con 0x7facf00b1250 2024-01-16T15:00:10.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.204+0000 7facf5800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facf015d990 con 0x7facf00ae310 2024-01-16T15:00:10.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.204+0000 7fcc44800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc3403e800 0x7fcc34039bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.204+0000 7fcc44800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc340542c0 0x7fcc3403a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.205+0000 7fcc44800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc340e3080 0x7fcc340e4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.205+0000 7fcc44800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc340502d0 con 0x7fcc340e3080 2024-01-16T15:00:10.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.205+0000 7fcc44800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc34179a40 con 0x7fcc3403e800 2024-01-16T15:00:10.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.205+0000 7fcc44800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc34179d20 con 0x7fcc340542c0 2024-01-16T15:00:10.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.205+0000 7f9084000640 1 Processor -- start 2024-01-16T15:00:10.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.205+0000 7f9084000640 1 -- start start 2024-01-16T15:00:10.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.205+0000 7facef400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7facf00b1250 0x7facf00cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.205+0000 7fcc42800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc340e3080 0x7fcc340e4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.205+0000 7facef400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7facf00b1250 0x7facf00cd460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42504/0 (socket says 172.21.15.143:42504) 2024-01-16T15:00:10.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.205+0000 7f9a7e800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a780b1190 0x7f9a781524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.205+0000 7f9a7e800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a7815d500 0x7f9a78151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.205+0000 7f9a7e800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a78154a20 0x7f9a780b6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.205+0000 7fcc41400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc340542c0 0x7fcc3403a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.205+0000 7fcc41400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc340542c0 0x7fcc3403a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:57394/0 (socket says 172.21.15.143:57394) 2024-01-16T15:00:10.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.205+0000 7fcc41400640 1 -- 172.21.15.143:0/328482407 learned_addr learned my addr 172.21.15.143:0/328482407 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:10.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.205+0000 7facef400640 1 -- 172.21.15.143:0/257412259 learned_addr learned my addr 172.21.15.143:0/257412259 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:10.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.206+0000 7f9a7e800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a780c9dd0 con 0x7f9a780b1190 2024-01-16T15:00:10.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.206+0000 7f9a7e800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a78152b90 con 0x7f9a78154a20 2024-01-16T15:00:10.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.206+0000 7f9a7e800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a78152e70 con 0x7f9a7815d500 2024-01-16T15:00:10.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.206+0000 7f9a7d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a780b1190 0x7f9a781524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.206+0000 7f9a7d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a780b1190 0x7f9a781524b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42516/0 (socket says 172.21.15.143:42516) 2024-01-16T15:00:10.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.206+0000 7fcc41e00640 1 --2- 172.21.15.143:0/328482407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc3403e800 0x7fcc34039bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.205+0000 7f1560a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f15540e3190 0x7f1554006c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.205+0000 7f1560a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1554007160 0x7f15540e6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.206+0000 7f1560a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15540e7780 con 0x7f15540e7330 2024-01-16T15:00:10.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.206+0000 7f1560a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15540570c0 con 0x7f15540e3190 2024-01-16T15:00:10.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.206+0000 7f1560a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1554057640 con 0x7f1554007160 2024-01-16T15:00:10.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.206+0000 7f9a7d400640 1 -- 172.21.15.143:0/2873695655 learned_addr learned my addr 172.21.15.143:0/2873695655 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:10.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.206+0000 7ff1a7e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1a0053c10 0x7ff1a003bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.205+0000 7f9429400640 1 --2- 172.21.15.143:0/1890630576 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94240ce270 0x7f94240d3770 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f941c009d90 tx=0x7f941c02beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fa6531de29e1b71 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.206+0000 7f982d400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f982815d490 0x7f98280cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.206+0000 7facf4e00640 1 --2- 172.21.15.143:0/257412259 >> v2:172.21.15.204:3300/0 conn(0x7facf00ae310 0x7facf00cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.206+0000 7ff1a7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1a0058990 0x7ff1a00e05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.207+0000 7f9a7de00640 1 --2- 172.21.15.143:0/2873695655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a78154a20 0x7f9a780b6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.206+0000 7fcc42800640 1 -- 172.21.15.143:0/328482407 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc340542c0 msgr2=0x7fcc3403a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.206+0000 7fcc42800640 1 --2- 172.21.15.143:0/328482407 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc340542c0 0x7fcc3403a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.206+0000 7fcc42800640 1 -- 172.21.15.143:0/328482407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc3403e800 msgr2=0x7fcc34039bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.206+0000 7fcc42800640 1 --2- 172.21.15.143:0/328482407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc3403e800 0x7fcc34039bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.206+0000 7fcc42800640 1 -- 172.21.15.143:0/328482407 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc3c04f730 con 0x7fcc340e3080 2024-01-16T15:00:10.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.206+0000 7fcc41e00640 1 --2- 172.21.15.143:0/328482407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc3403e800 0x7fcc34039bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:10.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.206+0000 7ff1a7e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1a00e76e0 0x7ff1a00e0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.207+0000 7f9423400640 1 -- 172.21.15.143:0/1890630576 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c02cc60 con 0x7f94240ce270 2024-01-16T15:00:10.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.207+0000 7f9a7ca00640 1 --2- 172.21.15.143:0/2873695655 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a7815d500 0x7f9a78151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.207+0000 7f155aa00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1554007160 0x7f15540e6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.207+0000 7f982d400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98280cea30 con 0x7f98280b63d0 2024-01-16T15:00:10.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.207+0000 7f982d400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98280ce750 con 0x7f98280d3940 2024-01-16T15:00:10.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.207+0000 7f9423400640 1 -- 172.21.15.143:0/1890630576 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f941c02cdc0 con 0x7f94240ce270 2024-01-16T15:00:10.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.207+0000 7f9423400640 1 -- 172.21.15.143:0/1890630576 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c034660 con 0x7f94240ce270 2024-01-16T15:00:10.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.207+0000 7f155aa00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1554007160 0x7f15540e6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:60538/0 (socket says 172.21.15.143:60538) 2024-01-16T15:00:10.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.207+0000 7f155aa00640 1 -- 172.21.15.143:0/4168836724 learned_addr learned my addr 172.21.15.143:0/4168836724 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:10.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.206+0000 7ff1a7e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a00dfb40 con 0x7ff1a00e76e0 2024-01-16T15:00:10.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.207+0000 7f982d400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98280d0450 con 0x7f982815d490 2024-01-16T15:00:10.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.206+0000 7ff1a7e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a00e95a0 con 0x7ff1a0058990 2024-01-16T15:00:10.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.206+0000 7ff1a7e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a00e92c0 con 0x7ff1a0053c10 2024-01-16T15:00:10.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.207+0000 7ff1a6a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1a0053c10 0x7ff1a003bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.207+0000 7ff1a6a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1a0053c10 0x7ff1a003bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:57410/0 (socket says 172.21.15.143:57410) 2024-01-16T15:00:10.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.207+0000 7ff1a6a00640 1 -- 172.21.15.143:0/1772517210 learned_addr learned my addr 172.21.15.143:0/1772517210 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:10.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.207+0000 7ff1a6000640 1 --2- 172.21.15.143:0/1772517210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1a0058990 0x7ff1a00e05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.207+0000 7f9827e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f982815d490 0x7f98280cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.207+0000 7f9827e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f982815d490 0x7f98280cd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:57416/0 (socket says 172.21.15.143:57416) 2024-01-16T15:00:10.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.207+0000 7fcc42800640 1 --2- 172.21.15.143:0/328482407 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc340e3080 0x7fcc340e4720 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcc3403ecb0 tx=0x7fcc2c00d9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.207+0000 7f9a7d400640 1 -- 172.21.15.143:0/2873695655 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a7815d500 msgr2=0x7f9a78151ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.207+0000 7f9a7d400640 1 --2- 172.21.15.143:0/2873695655 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a7815d500 0x7f9a78151ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.207+0000 7f9a7d400640 1 -- 172.21.15.143:0/2873695655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a78154a20 msgr2=0x7f9a780b6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.207+0000 7f9a7d400640 1 --2- 172.21.15.143:0/2873695655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a78154a20 0x7f9a780b6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.207+0000 7f981fe00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f98280b63d0 0x7f98280d3230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54116/0 (socket says 172.21.15.143:54116) 2024-01-16T15:00:10.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.207+0000 7f981fe00640 1 -- 172.21.15.143:0/85631962 learned_addr learned my addr 172.21.15.143:0/85631962 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:10.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.207+0000 7f9a7d400640 1 -- 172.21.15.143:0/2873695655 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a680094d0 con 0x7f9a780b1190 2024-01-16T15:00:10.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.207+0000 7ff1a7400640 1 --2- 172.21.15.143:0/1772517210 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1a00e76e0 0x7ff1a00e0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.207+0000 7f155b400640 1 --2- 172.21.15.143:0/4168836724 >> v2:172.21.15.143:3300/0 conn(0x7f15540e3190 0x7f1554006c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.208+0000 7f9429e00640 1 -- 172.21.15.143:0/1890630576 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94240ce270 msgr2=0x7f94240d3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.208+0000 7f9429e00640 1 --2- 172.21.15.143:0/1890630576 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94240ce270 0x7f94240d3770 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f941c009d90 tx=0x7f941c02beb0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.208+0000 7f9084000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f907403e840 0x7f9074053bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.208+0000 7f9084000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9074054130 0x7f90740068d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.208+0000 7f9084000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9074058570 con 0x7f90740e30c0 2024-01-16T15:00:10.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.208+0000 7f9084000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90740545c0 con 0x7f9074054130 2024-01-16T15:00:10.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.208+0000 7f9084000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90740061a0 con 0x7f907403e840 2024-01-16T15:00:10.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.208+0000 7f155aa00640 1 -- 172.21.15.143:0/4168836724 >> v2:172.21.15.143:3300/0 conn(0x7f15540e3190 msgr2=0x7f1554006c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:10.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.208+0000 7f155aa00640 1 --2- 172.21.15.143:0/4168836724 >> v2:172.21.15.143:3300/0 conn(0x7f15540e3190 0x7f1554006c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.208+0000 7f155aa00640 1 -- 172.21.15.143:0/4168836724 >> v1:172.21.15.204:6789/0 conn(0x7f15540e7330 legacy=0x7f1554053770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:10.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.208+0000 7f155aa00640 1 -- 172.21.15.143:0/4168836724 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15540076c0 con 0x7f1554007160 2024-01-16T15:00:10.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.208+0000 7f155b400640 1 --2- 172.21.15.143:0/4168836724 >> v2:172.21.15.143:3300/0 conn(0x7f15540e3190 0x7f1554006c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:10.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.208+0000 7f9081600640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f90740e30c0 0x7f9074057d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54126/0 (socket says 172.21.15.143:54126) 2024-01-16T15:00:10.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.208+0000 7f9081600640 1 -- 172.21.15.143:0/3028736823 learned_addr learned my addr 172.21.15.143:0/3028736823 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:10.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.208+0000 7f9429e00640 1 -- 172.21.15.143:0/1890630576 shutdown_connections 2024-01-16T15:00:10.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.208+0000 7f9429e00640 1 --2- 172.21.15.143:0/1890630576 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94240ce270 0x7f94240d3770 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.208+0000 7f9429e00640 1 --2- 172.21.15.143:0/1890630576 >> v2:172.21.15.204:3300/0 conn(0x7f94240b63d0 0x7f94240d3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.208+0000 7f9429e00640 1 -- 172.21.15.143:0/1890630576 >> 172.21.15.143:0/1890630576 conn(0x7f9424061340 msgr2=0x7f9424061750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:10.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.208+0000 7f9a7d400640 1 --2- 172.21.15.143:0/2873695655 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a780b1190 0x7f9a781524b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9a6000c420 tx=0x7f9a6000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.208+0000 7f9080c00640 1 --2- 172.21.15.143:0/3028736823 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f907403e840 0x7f9074053bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.207+0000 7facef400640 1 -- 172.21.15.143:0/257412259 >> v2:172.21.15.204:3300/0 conn(0x7facf00ae310 msgr2=0x7facf00cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:10.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.207+0000 7facef400640 1 --2- 172.21.15.143:0/257412259 >> v2:172.21.15.204:3300/0 conn(0x7facf00ae310 0x7facf00cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.207+0000 7facef400640 1 -- 172.21.15.143:0/257412259 >> v1:172.21.15.204:6789/0 conn(0x7facf0159670 legacy=0x7facf00b0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.207+0000 7facef400640 1 -- 172.21.15.143:0/257412259 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf015dc70 con 0x7facf00b1250 2024-01-16T15:00:10.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.209+0000 7f9429e00640 1 -- 172.21.15.143:0/1890630576 shutdown_connections 2024-01-16T15:00:10.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.209+0000 7f9429e00640 1 -- 172.21.15.143:0/1890630576 wait complete. 2024-01-16T15:00:10.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.209+0000 7f9827400640 1 -- 172.21.15.143:0/85631962 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2898729705 0 0) 0x7f98280cea30 con 0x7f98280b63d0 2024-01-16T15:00:10.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.209+0000 7ff1a6a00640 1 -- 172.21.15.143:0/1772517210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1a0058990 msgr2=0x7ff1a00e05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.209+0000 7ff1a6a00640 1 --2- 172.21.15.143:0/1772517210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1a0058990 0x7ff1a00e05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.209+0000 7ff1a6a00640 1 -- 172.21.15.143:0/1772517210 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1a00e76e0 msgr2=0x7ff1a00e0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.209+0000 7ff1a6a00640 1 --2- 172.21.15.143:0/1772517210 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1a00e76e0 0x7ff1a00e0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.209+0000 7f9827e00640 1 -- 172.21.15.143:0/85631962 >> v1:172.21.15.204:6789/0 conn(0x7f98280d3940 legacy=0x7f98280d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:10.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.209+0000 7ff1a6a00640 1 -- 172.21.15.143:0/1772517210 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a804f730 con 0x7ff1a0053c10 2024-01-16T15:00:10.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.209+0000 7ff1a7400640 1 --2- 172.21.15.143:0/1772517210 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1a00e76e0 0x7ff1a00e0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T15:00:10.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.209+0000 7f9827e00640 1 -- 172.21.15.143:0/85631962 >> v1:172.21.15.143:6789/0 conn(0x7f98280b63d0 legacy=0x7f98280d3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.209+0000 7fcc3a000640 1 -- 172.21.15.143:0/328482407 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c0125f0 con 0x7fcc340e3080 2024-01-16T15:00:10.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.209+0000 7fcc3a000640 1 -- 172.21.15.143:0/328482407 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc2c012750 con 0x7fcc340e3080 2024-01-16T15:00:10.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.209+0000 7fcc3a000640 1 -- 172.21.15.143:0/328482407 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c012ae0 con 0x7fcc340e3080 2024-01-16T15:00:10.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.209+0000 7f9827e00640 1 -- 172.21.15.143:0/85631962 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98280d0630 con 0x7f982815d490 2024-01-16T15:00:10.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.209+0000 7fcc44800640 1 -- 172.21.15.143:0/328482407 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc3417a000 con 0x7fcc340e3080 2024-01-16T15:00:10.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.210+0000 7f9a75600640 1 -- 172.21.15.143:0/2873695655 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a600074e0 con 0x7f9a780b1190 2024-01-16T15:00:10.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.210+0000 7ff1a6a00640 1 --2- 172.21.15.143:0/1772517210 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1a0053c10 0x7ff1a003bad0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff19800c390 tx=0x7ff19800c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.210+0000 7f9a75600640 1 -- 172.21.15.143:0/2873695655 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a60007640 con 0x7f9a780b1190 2024-01-16T15:00:10.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.210+0000 7f9a75600640 1 -- 172.21.15.143:0/2873695655 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a60004a20 con 0x7f9a780b1190 2024-01-16T15:00:10.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.210+0000 7f9a7e800640 1 -- 172.21.15.143:0/2873695655 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a78153150 con 0x7f9a780b1190 2024-01-16T15:00:10.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.209+0000 7fcc44800640 1 -- 172.21.15.143:0/328482407 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc3404eed0 con 0x7fcc340e3080 2024-01-16T15:00:10.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.210+0000 7ff19ea00640 1 -- 172.21.15.143:0/1772517210 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff198005ce0 con 0x7ff1a0053c10 2024-01-16T15:00:10.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.210+0000 7facef400640 1 --2- 172.21.15.143:0/257412259 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facf00b1250 0x7facf00cd460 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7facf00cebe0 tx=0x7facd802de00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ae8a082917a07f server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.209+0000 7f9082000640 1 --2- 172.21.15.143:0/3028736823 >> v2:172.21.15.204:3300/0 conn(0x7f9074054130 0x7f90740068d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.210+0000 7f9080c00640 1 -- 172.21.15.143:0/3028736823 >> v2:172.21.15.204:3300/0 conn(0x7f9074054130 msgr2=0x7f90740068d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:10.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.209+0000 7f907be00640 1 -- 172.21.15.143:0/3028736823 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2245447669 0 0) 0x7f9074058570 con 0x7f90740e30c0 2024-01-16T15:00:10.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.209+0000 7f155aa00640 1 --2- 172.21.15.143:0/4168836724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1554007160 0x7f15540e6df0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f155403e8e0 tx=0x7f154402b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24055b3d273d3ee1 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.210+0000 7f155a000640 1 -- 172.21.15.143:0/4168836724 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1544030860 con 0x7f1554007160 2024-01-16T15:00:10.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.210+0000 7ff1a7e00640 1 -- 172.21.15.143:0/1772517210 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1a00dfde0 con 0x7ff1a0053c10 2024-01-16T15:00:10.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.210+0000 7ff1a7e00640 1 -- 172.21.15.143:0/1772517210 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a00db0d0 con 0x7ff1a0053c10 2024-01-16T15:00:10.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.211+0000 7f155a000640 1 -- 172.21.15.143:0/4168836724 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15440309c0 con 0x7f1554007160 2024-01-16T15:00:10.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.211+0000 7ff19ea00640 1 -- 172.21.15.143:0/1772517210 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff198005e40 con 0x7ff1a0053c10 2024-01-16T15:00:10.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.211+0000 7ff19ea00640 1 -- 172.21.15.143:0/1772517210 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff198005ce0 con 0x7ff1a0053c10 2024-01-16T15:00:10.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.211+0000 7f155a000640 1 -- 172.21.15.143:0/4168836724 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1544030d50 con 0x7f1554007160 2024-01-16T15:00:10.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.210+0000 7f9a7e800640 1 -- 172.21.15.143:0/2873695655 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a78153710 con 0x7f9a780b1190 2024-01-16T15:00:10.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.211+0000 7fcc44800640 1 -- 172.21.15.143:0/328482407 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcc340e0450 con 0x7fcc340e3080 2024-01-16T15:00:10.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.210+0000 7f9080c00640 1 --2- 172.21.15.143:0/3028736823 >> v2:172.21.15.204:3300/0 conn(0x7f9074054130 0x7f90740068d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.210+0000 7f9080c00640 1 -- 172.21.15.143:0/3028736823 >> v1:172.21.15.143:6789/0 conn(0x7f90740e30c0 legacy=0x7f9074057d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.210+0000 7f9080c00640 1 -- 172.21.15.143:0/3028736823 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9074058850 con 0x7f907403e840 2024-01-16T15:00:10.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.212+0000 7faceea00640 1 -- 172.21.15.143:0/257412259 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd802eb30 con 0x7facf00b1250 2024-01-16T15:00:10.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.212+0000 7f9a7e800640 1 -- 172.21.15.143:0/2873695655 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9a780b06b0 con 0x7f9a780b1190 2024-01-16T15:00:10.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.212+0000 7f9429e00640 1 Processor -- start 2024-01-16T15:00:10.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.212+0000 7faceea00640 1 -- 172.21.15.143:0/257412259 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd8032070 con 0x7facf00b1250 2024-01-16T15:00:10.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.212+0000 7faceea00640 1 -- 172.21.15.143:0/257412259 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd8038400 con 0x7facf00b1250 2024-01-16T15:00:10.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.212+0000 7ff1a7e00640 1 -- 172.21.15.143:0/1772517210 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff178000fc0 con 0x7ff1a0053c10 2024-01-16T15:00:10.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.212+0000 7f9429e00640 1 -- start start 2024-01-16T15:00:10.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.212+0000 7f9827e00640 1 --2- 172.21.15.143:0/85631962 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f982815d490 0x7f98280cd8f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f98180097d0 tx=0x7f981802b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4a547fb0b9faa2a server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.212+0000 7f1560a00640 1 -- 172.21.15.143:0/4168836724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1554007160 msgr2=0x7f15540e6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.212+0000 7f1560a00640 1 --2- 172.21.15.143:0/4168836724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1554007160 0x7f15540e6df0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f155403e8e0 tx=0x7f154402b9a0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.213+0000 7f1560a00640 1 -- 172.21.15.143:0/4168836724 shutdown_connections 2024-01-16T15:00:10.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.213+0000 7f1560a00640 1 --2- 172.21.15.143:0/4168836724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1554007160 0x7f15540e6df0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.213+0000 7f1560a00640 1 --2- 172.21.15.143:0/4168836724 >> v2:172.21.15.143:3300/0 conn(0x7f15540e3190 0x7f1554006c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.213+0000 7f1560a00640 1 -- 172.21.15.143:0/4168836724 >> 172.21.15.143:0/4168836724 conn(0x7f155403aa00 msgr2=0x7f155403ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:10.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.213+0000 7f9827400640 1 -- 172.21.15.143:0/85631962 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9818031800 con 0x7f982815d490 2024-01-16T15:00:10.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.213+0000 7f9080c00640 1 --2- 172.21.15.143:0/3028736823 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f907403e840 0x7f9074053bf0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f90640097a0 tx=0x7f906402b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=472c5623c2e6605d server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.213+0000 7f9827400640 1 -- 172.21.15.143:0/85631962 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9818031960 con 0x7f982815d490 2024-01-16T15:00:10.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.213+0000 7f1560a00640 1 -- 172.21.15.143:0/4168836724 shutdown_connections 2024-01-16T15:00:10.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.213+0000 7facf5800640 1 -- 172.21.15.143:0/257412259 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facf00b1250 msgr2=0x7facf00cd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.213+0000 7facf5800640 1 --2- 172.21.15.143:0/257412259 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facf00b1250 0x7facf00cd460 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7facf00cebe0 tx=0x7facd802de00 comp rx=0 tx=0).stop 2024-01-16T15:00:10.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.213+0000 7facf5800640 1 -- 172.21.15.143:0/257412259 shutdown_connections 2024-01-16T15:00:10.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.213+0000 7facf5800640 1 --2- 172.21.15.143:0/257412259 >> v2:172.21.15.204:3300/0 conn(0x7facf00ae310 0x7facf00cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.213+0000 7facf5800640 1 --2- 172.21.15.143:0/257412259 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facf00b1250 0x7facf00cd460 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.213+0000 7facf5800640 1 -- 172.21.15.143:0/257412259 >> 172.21.15.143:0/257412259 conn(0x7facf00cbb40 msgr2=0x7facf00cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:10.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.214+0000 7f9429e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94240ce270 0x7f9424160120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.214+0000 7f9429e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f942415d490 0x7f94240cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.214+0000 7f9429e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94240ca370 0x7f94240cb230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.214+0000 7f9429e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94240b71d0 con 0x7f94240ca370 2024-01-16T15:00:10.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.214+0000 7f9429e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94240b7a00 con 0x7f94240ce270 2024-01-16T15:00:10.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.214+0000 7f9429e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f942415e670 con 0x7f942415d490 2024-01-16T15:00:10.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.214+0000 7f907be00640 1 -- 172.21.15.143:0/3028736823 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9064030610 con 0x7f907403e840 2024-01-16T15:00:10.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.214+0000 7f907be00640 1 -- 172.21.15.143:0/3028736823 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9064030770 con 0x7f907403e840 2024-01-16T15:00:10.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.214+0000 7f907be00640 1 -- 172.21.15.143:0/3028736823 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9064030b40 con 0x7f907403e840 2024-01-16T15:00:10.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.213+0000 7f1560a00640 1 -- 172.21.15.143:0/4168836724 wait complete. 2024-01-16T15:00:10.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.214+0000 7f982d400640 1 -- 172.21.15.143:0/85631962 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f982815d490 msgr2=0x7f98280cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.214+0000 7f982d400640 1 --2- 172.21.15.143:0/85631962 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f982815d490 0x7f98280cd8f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f98180097d0 tx=0x7f981802b8a0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.214+0000 7f982d400640 1 -- 172.21.15.143:0/85631962 shutdown_connections 2024-01-16T15:00:10.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.214+0000 7f982d400640 1 --2- 172.21.15.143:0/85631962 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f982815d490 0x7f98280cd8f0 secure :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f98180097d0 tx=0x7f981802b8a0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.215+0000 7f982d400640 1 -- 172.21.15.143:0/85631962 >> 172.21.15.143:0/85631962 conn(0x7f9828061340 msgr2=0x7f9828061750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:10.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.215+0000 7f982d400640 1 -- 172.21.15.143:0/85631962 shutdown_connections 2024-01-16T15:00:10.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.215+0000 7ff19ea00640 1 -- 172.21.15.143:0/1772517210 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7ff198007070 con 0x7ff1a0053c10 2024-01-16T15:00:10.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.215+0000 7f982d400640 1 -- 172.21.15.143:0/85631962 wait complete. 2024-01-16T15:00:10.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.214+0000 7fcc3a000640 1 -- 172.21.15.143:0/328482407 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fcc2c012c40 con 0x7fcc340e3080 2024-01-16T15:00:10.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.215+0000 7fcc3a000640 1 --2- 172.21.15.143:0/328482407 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fcc1406b390 0x7fcc1406d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.215+0000 7fcc3a000640 1 -- 172.21.15.143:0/328482407 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcc2c08a030 con 0x7fcc340e3080 2024-01-16T15:00:10.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.215+0000 7fcc3a000640 1 --2- 172.21.15.143:0/328482407 >> [v2:172.21.15.143:6802/1276841168,v1:172.21.15.143:6804/1276841168] conn(0x7fcc14073530 0x7fcc14075950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.215+0000 7facf5800640 1 -- 172.21.15.143:0/257412259 shutdown_connections 2024-01-16T15:00:10.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.215+0000 7fcc3a000640 1 -- 172.21.15.143:0/328482407 --> [v2:172.21.15.143:6802/1276841168,v1:172.21.15.143:6804/1276841168] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcc14076000 con 0x7fcc14073530 2024-01-16T15:00:10.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.215+0000 7fcc3a000640 1 -- 172.21.15.143:0/328482407 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc2c012ef0 con 0x7fcc340e3080 2024-01-16T15:00:10.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.215+0000 7facf5800640 1 -- 172.21.15.143:0/257412259 wait complete. 2024-01-16T15:00:10.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.215+0000 7f1560a00640 1 Processor -- start 2024-01-16T15:00:10.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.215+0000 7f9084000640 1 -- 172.21.15.143:0/3028736823 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f907403e840 msgr2=0x7f9074053bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.216+0000 7f1560a00640 1 -- start start 2024-01-16T15:00:10.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.216+0000 7f9084000640 1 --2- 172.21.15.143:0/3028736823 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f907403e840 0x7f9074053bf0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f90640097a0 tx=0x7f906402b960 comp rx=0 tx=0).stop 2024-01-16T15:00:10.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.216+0000 7f9429400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94240ca370 0x7f94240cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.216+0000 7fcc41400640 1 --2- 172.21.15.143:0/328482407 >> [v2:172.21.15.143:6802/1276841168,v1:172.21.15.143:6804/1276841168] conn(0x7fcc14073530 0x7fcc14075950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.216+0000 7f9429400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94240ca370 0x7f94240cb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42544/0 (socket says 172.21.15.143:42544) 2024-01-16T15:00:10.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.216+0000 7f9429400640 1 -- 172.21.15.143:0/1038860075 learned_addr learned my addr 172.21.15.143:0/1038860075 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:10.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.216+0000 7f9428a00640 1 --2- 172.21.15.143:0/1038860075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94240ce270 0x7f9424160120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.216+0000 7f9084000640 1 -- 172.21.15.143:0/3028736823 shutdown_connections 2024-01-16T15:00:10.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.216+0000 7f9084000640 1 --2- 172.21.15.143:0/3028736823 >> v2:172.21.15.204:3300/0 conn(0x7f9074054130 0x7f90740068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.216+0000 7f9084000640 1 --2- 172.21.15.143:0/3028736823 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f907403e840 0x7f9074053bf0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.216+0000 7f9084000640 1 -- 172.21.15.143:0/3028736823 >> 172.21.15.143:0/3028736823 conn(0x7f90740559e0 msgr2=0x7f9074055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:10.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.216+0000 7fcc41e00640 1 --2- 172.21.15.143:0/328482407 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fcc1406b390 0x7fcc1406d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.216+0000 7f9a75600640 1 -- 172.21.15.143:0/2873695655 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f9a60004b80 con 0x7f9a780b1190 2024-01-16T15:00:10.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.217+0000 7f9423e00640 1 --2- 172.21.15.143:0/1038860075 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f942415d490 0x7f94240cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.217+0000 7f9084000640 1 -- 172.21.15.143:0/3028736823 shutdown_connections 2024-01-16T15:00:10.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.217+0000 7fcc41400640 1 --2- 172.21.15.143:0/328482407 >> [v2:172.21.15.143:6802/1276841168,v1:172.21.15.143:6804/1276841168] conn(0x7fcc14073530 0x7fcc14075950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.217+0000 7fcc41e00640 1 --2- 172.21.15.143:0/328482407 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fcc1406b390 0x7fcc1406d850 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcc340085f0 tx=0x7fcc3c0ab000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.217+0000 7f9428a00640 1 -- 172.21.15.143:0/1038860075 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f942415d490 msgr2=0x7f94240cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.217+0000 7f9428a00640 1 --2- 172.21.15.143:0/1038860075 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f942415d490 0x7f94240cacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.217+0000 7f9428a00640 1 -- 172.21.15.143:0/1038860075 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94240ca370 msgr2=0x7f94240cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.218+0000 7f9084000640 1 -- 172.21.15.143:0/3028736823 wait complete. 2024-01-16T15:00:10.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.218+0000 7f1560a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1554007160 0x7f155404c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.218+0000 7f9a75600640 1 --2- 172.21.15.143:0/2873695655 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f9a5006b3c0 0x7f9a5006d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.218+0000 7f1560a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15540e3190 0x7f1554051630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.218+0000 7f9428a00640 1 --2- 172.21.15.143:0/1038860075 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94240ca370 0x7f94240cb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.218+0000 7facf5800640 1 Processor -- start 2024-01-16T15:00:10.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.218+0000 7f9428a00640 1 -- 172.21.15.143:0/1038860075 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f941c009a70 con 0x7f94240ce270 2024-01-16T15:00:10.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.218+0000 7f1560a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15540e7330 0x7f15540d7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.218+0000 7f155b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1554007160 0x7f155404c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.218+0000 7f9429400640 1 --2- 172.21.15.143:0/1038860075 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94240ca370 0x7f94240cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T15:00:10.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.218+0000 7f155b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1554007160 0x7f155404c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42552/0 (socket says 172.21.15.143:42552) 2024-01-16T15:00:10.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.218+0000 7f9a75600640 1 -- 172.21.15.143:0/2873695655 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9a60089c50 con 0x7f9a780b1190 2024-01-16T15:00:10.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.218+0000 7f1560a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155404dc30 con 0x7f1554007160 2024-01-16T15:00:10.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.218+0000 7f1560a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155404cfb0 con 0x7f15540e3190 2024-01-16T15:00:10.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.218+0000 7f1560a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f155404d290 con 0x7f15540e7330 2024-01-16T15:00:10.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.217+0000 7f982d400640 1 Processor -- start 2024-01-16T15:00:10.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.218+0000 7f9428a00640 1 --2- 172.21.15.143:0/1038860075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94240ce270 0x7f9424160120 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f941400c410 tx=0x7f941400c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.218+0000 7f155b400640 1 -- 172.21.15.143:0/4282266046 learned_addr learned my addr 172.21.15.143:0/4282266046 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:10.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.219+0000 7f9a75600640 1 --2- 172.21.15.143:0/2873695655 >> [v2:172.21.15.143:6800/1569293626,v1:172.21.15.143:6801/1569293626] conn(0x7f9a500735e0 0x7f9a50075a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.219+0000 7f9a75600640 1 -- 172.21.15.143:0/2873695655 --> [v2:172.21.15.143:6800/1569293626,v1:172.21.15.143:6801/1569293626] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9a500760b0 con 0x7f9a500735e0 2024-01-16T15:00:10.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.219+0000 7f9a75600640 1 -- 172.21.15.143:0/2873695655 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a6008a000 con 0x7f9a780b1190 2024-01-16T15:00:10.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.219+0000 7f9a7ca00640 1 --2- 172.21.15.143:0/2873695655 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f9a5006b3c0 0x7f9a5006d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.219+0000 7f155be00640 1 --2- 172.21.15.143:0/4282266046 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15540e7330 0x7f15540d7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.219+0000 7fcc3a000640 1 -- 172.21.15.143:0/328482407 <== osd.1 v2:172.21.15.143:6802/1276841168 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fcc14076000 con 0x7fcc14073530 2024-01-16T15:00:10.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.219+0000 7facf5800640 1 -- start start 2024-01-16T15:00:10.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.219+0000 7f9a7de00640 1 --2- 172.21.15.143:0/2873695655 >> [v2:172.21.15.143:6800/1569293626,v1:172.21.15.143:6801/1569293626] conn(0x7f9a500735e0 0x7f9a50075a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.219+0000 7f155be00640 1 -- 172.21.15.143:0/4282266046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15540e3190 msgr2=0x7f1554051630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-16T15:00:10.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.219+0000 7f155be00640 1 --2- 172.21.15.143:0/4282266046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15540e3190 0x7f1554051630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.219+0000 7f155be00640 1 -- 172.21.15.143:0/4282266046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1554007160 msgr2=0x7f155404c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.220+0000 7facf5800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7facf00ae310 0x7facf00c8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.220+0000 7facf5800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facf00b1250 0x7facf00b3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.219+0000 7f155be00640 1 --2- 172.21.15.143:0/4282266046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1554007160 0x7f155404c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.219+0000 7f155be00640 1 -- 172.21.15.143:0/4282266046 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15440094d0 con 0x7f15540e7330 2024-01-16T15:00:10.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.220+0000 7f982d400640 1 -- start start 2024-01-16T15:00:10.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.220+0000 7facf5800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facf0159670 0x7facf0159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.220+0000 7facf5800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf00b45d0 con 0x7facf0159670 2024-01-16T15:00:10.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.220+0000 7facf5800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf00c66a0 con 0x7facf00b1250 2024-01-16T15:00:10.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.220+0000 7facf5800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf00b4010 con 0x7facf00ae310 2024-01-16T15:00:10.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.220+0000 7f155be00640 1 --2- 172.21.15.143:0/4282266046 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15540e7330 0x7f15540d7300 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f155c0733c0 tx=0x7f155c068010 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.219+0000 7ff19ea00640 1 --2- 172.21.15.143:0/1772517210 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7ff18006b320 0x7ff18006d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.219+0000 7ff19ea00640 1 -- 172.21.15.143:0/1772517210 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff198089570 con 0x7ff1a0053c10 2024-01-16T15:00:10.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.219+0000 7ff19ea00640 1 --2- 172.21.15.143:0/1772517210 >> [v2:172.21.15.143:6816/1217670170,v1:172.21.15.143:6817/1217670170] conn(0x7ff180073540 0x7ff180075960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.219+0000 7ff19ea00640 1 -- 172.21.15.143:0/1772517210 --> [v2:172.21.15.143:6816/1217670170,v1:172.21.15.143:6817/1217670170] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff180076010 con 0x7ff180073540 2024-01-16T15:00:10.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.219+0000 7ff19ea00640 1 -- 172.21.15.143:0/1772517210 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff198089920 con 0x7ff1a0053c10 2024-01-16T15:00:10.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.220+0000 7ff1a6000640 1 --2- 172.21.15.143:0/1772517210 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7ff18006b320 0x7ff18006d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.220+0000 7ff1a7400640 1 --2- 172.21.15.143:0/1772517210 >> [v2:172.21.15.143:6816/1217670170,v1:172.21.15.143:6817/1217670170] conn(0x7ff180073540 0x7ff180075960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.220+0000 7f9a7ca00640 1 --2- 172.21.15.143:0/2873695655 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f9a5006b3c0 0x7f9a5006d880 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9a68036000 tx=0x7f9a68008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.220+0000 7ff1a6000640 1 --2- 172.21.15.143:0/1772517210 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7ff18006b320 0x7ff18006d7e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff1a00dba90 tx=0x7ff190009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.221+0000 7facef400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facf00b1250 0x7facf00b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.221+0000 7facef400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facf00b1250 0x7facf00b3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:60598/0 (socket says 172.21.15.143:60598) 2024-01-16T15:00:10.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.221+0000 7facef400640 1 -- 172.21.15.143:0/210879554 learned_addr learned my addr 172.21.15.143:0/210879554 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:10.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.221+0000 7facefe00640 1 --2- 172.21.15.143:0/210879554 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7facf00ae310 0x7facf00c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.221+0000 7ff1a7400640 1 --2- 172.21.15.143:0/1772517210 >> [v2:172.21.15.143:6816/1217670170,v1:172.21.15.143:6817/1217670170] conn(0x7ff180073540 0x7ff180075960 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.221+0000 7facef400640 1 -- 172.21.15.143:0/210879554 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7facf00ae310 msgr2=0x7facf00c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.221+0000 7facef400640 1 --2- 172.21.15.143:0/210879554 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7facf00ae310 0x7facf00c8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.221+0000 7facef400640 1 -- 172.21.15.143:0/210879554 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facf0159670 msgr2=0x7facf0159bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.221+0000 7facef400640 1 --2- 172.21.15.143:0/210879554 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facf0159670 0x7facf0159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.221+0000 7facef400640 1 -- 172.21.15.143:0/210879554 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd8008550 con 0x7facf00b1250 2024-01-16T15:00:10.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.222+0000 7f982d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98280d3940 0x7f98281fa560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.222+0000 7f982d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98280c84c0 0x7f982815f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.222+0000 7f982d400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98280b77c0 0x7f98280b7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.222+0000 7f982d400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98280d0bf0 con 0x7f98280d3940 2024-01-16T15:00:10.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.222+0000 7f982d400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9828160910 con 0x7f98280c84c0 2024-01-16T15:00:10.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.222+0000 7f982d400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98280b8180 con 0x7f98280b77c0 2024-01-16T15:00:10.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.222+0000 7facef400640 1 --2- 172.21.15.143:0/210879554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facf00b1250 0x7facf00b3630 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7facd803a040 tx=0x7facd8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.222+0000 7ff19ea00640 1 -- 172.21.15.143:0/1772517210 <== osd.2 v2:172.21.15.143:6816/1217670170 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7ff180076010 con 0x7ff180073540 2024-01-16T15:00:10.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.223+0000 7f982ca00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98280b77c0 0x7f98280b7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.223+0000 7f982ca00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98280b77c0 0x7f98280b7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:57478/0 (socket says 172.21.15.143:57478) 2024-01-16T15:00:10.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.223+0000 7f982ca00640 1 -- 172.21.15.143:0/3706540023 learned_addr learned my addr 172.21.15.143:0/3706540023 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:10.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.222+0000 7face7e00640 1 -- 172.21.15.143:0/210879554 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd800b360 con 0x7facf00b1250 2024-01-16T15:00:10.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.222+0000 7face7e00640 1 -- 172.21.15.143:0/210879554 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd800b4c0 con 0x7facf00b1250 2024-01-16T15:00:10.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.222+0000 7face7e00640 1 -- 172.21.15.143:0/210879554 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd800b870 con 0x7facf00b1250 2024-01-16T15:00:10.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.223+0000 7facf5800640 1 -- 172.21.15.143:0/210879554 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facf00b42f0 con 0x7facf00b1250 2024-01-16T15:00:10.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.223+0000 7f981fe00640 1 --2- 172.21.15.143:0/3706540023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98280c84c0 0x7f982815f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.223+0000 7facf5800640 1 -- 172.21.15.143:0/210879554 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facf00c8770 con 0x7facf00b1250 2024-01-16T15:00:10.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.223+0000 7f1553400640 1 -- 172.21.15.143:0/4282266046 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c0721c0 con 0x7f15540e7330 2024-01-16T15:00:10.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.223+0000 7f1553400640 1 -- 172.21.15.143:0/4282266046 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f155c072320 con 0x7f15540e7330 2024-01-16T15:00:10.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.223+0000 7f1553400640 1 -- 172.21.15.143:0/4282266046 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c072790 con 0x7f15540e7330 2024-01-16T15:00:10.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.223+0000 7f9084000640 1 Processor -- start 2024-01-16T15:00:10.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.219+0000 7f9420c00640 1 -- 172.21.15.143:0/1038860075 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94140074e0 con 0x7f94240ce270 2024-01-16T15:00:10.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.219+0000 7f9420c00640 1 -- 172.21.15.143:0/1038860075 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9414007640 con 0x7f94240ce270 2024-01-16T15:00:10.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.219+0000 7f9420c00640 1 -- 172.21.15.143:0/1038860075 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9414004a20 con 0x7f94240ce270 2024-01-16T15:00:10.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.219+0000 7f9429e00640 1 -- 172.21.15.143:0/1038860075 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9424151d50 con 0x7f94240ce270 2024-01-16T15:00:10.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.220+0000 7f9429e00640 1 -- 172.21.15.143:0/1038860075 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94240b7690 con 0x7f94240ce270 2024-01-16T15:00:10.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.222+0000 7f9429e00640 1 -- 172.21.15.143:0/1038860075 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f94240d0910 con 0x7f94240ce270 2024-01-16T15:00:10.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.224+0000 7facf5800640 1 -- 172.21.15.143:0/210879554 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7facf00cee00 con 0x7facf00b1250 2024-01-16T15:00:10.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.223+0000 7f982ca00640 1 -- 172.21.15.143:0/3706540023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98280c84c0 msgr2=0x7f982815f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.223+0000 7f982ca00640 1 --2- 172.21.15.143:0/3706540023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98280c84c0 0x7f982815f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.223+0000 7f982ca00640 1 -- 172.21.15.143:0/3706540023 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98280d3940 msgr2=0x7f98281fa560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.223+0000 7f982ca00640 1 --2- 172.21.15.143:0/3706540023 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98280d3940 0x7f98281fa560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.223+0000 7f982ca00640 1 -- 172.21.15.143:0/3706540023 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9818009400 con 0x7f98280b77c0 2024-01-16T15:00:10.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.224+0000 7f982ca00640 1 --2- 172.21.15.143:0/3706540023 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98280b77c0 0x7f98280b7c40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f982000c6d0 tx=0x7f982000cb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.224+0000 7f9824c00640 1 -- 172.21.15.143:0/3706540023 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982000a840 con 0x7f98280b77c0 2024-01-16T15:00:10.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.224+0000 7f9824c00640 1 -- 172.21.15.143:0/3706540023 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f982000a9a0 con 0x7f98280b77c0 2024-01-16T15:00:10.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.224+0000 7f9824c00640 1 -- 172.21.15.143:0/3706540023 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982000ad90 con 0x7f98280b77c0 2024-01-16T15:00:10.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.224+0000 7f982d400640 1 -- 172.21.15.143:0/3706540023 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9828160480 con 0x7f98280b77c0 2024-01-16T15:00:10.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.224+0000 7f1560a00640 1 -- 172.21.15.143:0/4282266046 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f155404d570 con 0x7f15540e7330 2024-01-16T15:00:10.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.224+0000 7f1560a00640 1 -- 172.21.15.143:0/4282266046 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1554051b70 con 0x7f15540e7330 2024-01-16T15:00:10.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.224+0000 7f982d400640 1 -- 172.21.15.143:0/3706540023 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f982815fb50 con 0x7f98280b77c0 2024-01-16T15:00:10.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.225+0000 7f9a7de00640 1 --2- 172.21.15.143:0/2873695655 >> [v2:172.21.15.143:6800/1569293626,v1:172.21.15.143:6801/1569293626] conn(0x7f9a500735e0 0x7f9a50075a00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.226+0000 7f9a75600640 1 -- 172.21.15.143:0/2873695655 <== osd.0 v2:172.21.15.143:6800/1569293626 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f9a500760b0 con 0x7f9a500735e0 2024-01-16T15:00:10.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.226+0000 7f1560a00640 1 -- 172.21.15.143:0/4282266046 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f15540e7b60 con 0x7f15540e7330 2024-01-16T15:00:10.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.226+0000 7f9084000640 1 -- start start 2024-01-16T15:00:10.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.227+0000 7f982d400640 1 -- 172.21.15.143:0/3706540023 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f98280cde30 con 0x7f98280b77c0 2024-01-16T15:00:10.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.227+0000 7f9084000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f907403e840 0x7f90740d5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.227+0000 7f9084000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9074054130 0x7f90740d7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.227+0000 7f9084000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90740e30c0 0x7f90740de180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.227+0000 7f9084000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90740d5ec0 con 0x7f907403e840 2024-01-16T15:00:10.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.227+0000 7f9084000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90740dab30 con 0x7f90740e30c0 2024-01-16T15:00:10.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.227+0000 7f9084000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90740dae10 con 0x7f9074054130 2024-01-16T15:00:10.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.228+0000 7f9080c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9074054130 0x7f90740d7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.228+0000 7f9080c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9074054130 0x7f90740d7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:57480/0 (socket says 172.21.15.143:57480) 2024-01-16T15:00:10.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.228+0000 7f9080c00640 1 -- 172.21.15.143:0/1974852714 learned_addr learned my addr 172.21.15.143:0/1974852714 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:10.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.228+0000 7f9082000640 1 --2- 172.21.15.143:0/1974852714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90740e30c0 0x7f90740de180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.228+0000 7f9081600640 1 --2- 172.21.15.143:0/1974852714 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f907403e840 0x7f90740d5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.229+0000 7f9824c00640 1 -- 172.21.15.143:0/3706540023 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f9820012630 con 0x7f98280b77c0 2024-01-16T15:00:10.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.229+0000 7f9082000640 1 -- 172.21.15.143:0/1974852714 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9074054130 msgr2=0x7f90740d7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.229+0000 7f9082000640 1 --2- 172.21.15.143:0/1974852714 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9074054130 0x7f90740d7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.229+0000 7f9082000640 1 -- 172.21.15.143:0/1974852714 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f907403e840 msgr2=0x7f90740d5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.229+0000 7f9082000640 1 --2- 172.21.15.143:0/1974852714 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f907403e840 0x7f90740d5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.229+0000 7f9082000640 1 -- 172.21.15.143:0/1974852714 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9064009400 con 0x7f90740e30c0 2024-01-16T15:00:10.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.229+0000 7f9080c00640 1 --2- 172.21.15.143:0/1974852714 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9074054130 0x7f90740d7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T15:00:10.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.229+0000 7f9081600640 1 --2- 172.21.15.143:0/1974852714 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f907403e840 0x7f90740d5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T15:00:10.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.230+0000 7f9082000640 1 --2- 172.21.15.143:0/1974852714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90740e30c0 0x7f90740de180 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f907c064180 tx=0x7f907c068010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.230+0000 7f9824c00640 1 --2- 172.21.15.143:0/3706540023 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f980006b2c0 0x7f980006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.230+0000 7f9824c00640 1 -- 172.21.15.143:0/3706540023 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f982008a8a0 con 0x7f98280b77c0 2024-01-16T15:00:10.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.231+0000 7f9824c00640 1 --2- 172.21.15.143:0/3706540023 >> [v2:172.21.15.204:6811/1070255716,v1:172.21.15.204:6812/1070255716] conn(0x7f9800073460 0x7f9800075880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.231+0000 7f9824c00640 1 -- 172.21.15.143:0/3706540023 --> [v2:172.21.15.204:6811/1070255716,v1:172.21.15.204:6812/1070255716] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9800075f30 con 0x7f9800073460 2024-01-16T15:00:10.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.231+0000 7f9824c00640 1 -- 172.21.15.143:0/3706540023 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f982008ac10 con 0x7f98280b77c0 2024-01-16T15:00:10.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.231+0000 7f9827e00640 1 --2- 172.21.15.143:0/3706540023 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f980006b2c0 0x7f980006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.230+0000 7face7e00640 1 -- 172.21.15.143:0/210879554 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7facd8038400 con 0x7facf00b1250 2024-01-16T15:00:10.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.228+0000 7f1553400640 1 -- 172.21.15.143:0/4282266046 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f155c074940 con 0x7f15540e7330 2024-01-16T15:00:10.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.230+0000 7f1553400640 1 --2- 172.21.15.143:0/4282266046 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f153406b390 0x7f153406d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.230+0000 7f1553400640 1 -- 172.21.15.143:0/4282266046 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f155c103e20 con 0x7f15540e7330 2024-01-16T15:00:10.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.231+0000 7f1553400640 1 --2- 172.21.15.143:0/4282266046 >> [v2:172.21.15.204:6824/1337220609,v1:172.21.15.204:6825/1337220609] conn(0x7f15340735b0 0x7f15340759d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.231+0000 7f1553400640 1 -- 172.21.15.143:0/4282266046 --> [v2:172.21.15.204:6824/1337220609,v1:172.21.15.204:6825/1337220609] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1534076080 con 0x7f15340735b0 2024-01-16T15:00:10.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.231+0000 7f1553400640 1 -- 172.21.15.143:0/4282266046 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f155c1041d0 con 0x7f15540e7330 2024-01-16T15:00:10.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.231+0000 7f155b400640 1 --2- 172.21.15.143:0/4282266046 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f153406b390 0x7f153406d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.230+0000 7f9420c00640 1 -- 172.21.15.143:0/1038860075 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f9414004b80 con 0x7f94240ce270 2024-01-16T15:00:10.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.231+0000 7f9420c00640 1 --2- 172.21.15.143:0/1038860075 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f93fc06b340 0x7f93fc06d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.232+0000 7f155aa00640 1 --2- 172.21.15.143:0/4282266046 >> [v2:172.21.15.204:6824/1337220609,v1:172.21.15.204:6825/1337220609] conn(0x7f15340735b0 0x7f15340759d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.232+0000 7f9420c00640 1 -- 172.21.15.143:0/1038860075 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9414089f50 con 0x7f94240ce270 2024-01-16T15:00:10.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.232+0000 7f9420c00640 1 --2- 172.21.15.143:0/1038860075 >> [v2:172.21.15.143:6824/3414188012,v1:172.21.15.143:6825/3414188012] conn(0x7f93fc073560 0x7f93fc075980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.232+0000 7f9420c00640 1 -- 172.21.15.143:0/1038860075 --> [v2:172.21.15.143:6824/3414188012,v1:172.21.15.143:6825/3414188012] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f93fc076030 con 0x7f93fc073560 2024-01-16T15:00:10.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.232+0000 7f9420c00640 1 -- 172.21.15.143:0/1038860075 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f941408a3f0 con 0x7f94240ce270 2024-01-16T15:00:10.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.232+0000 7f9423e00640 1 --2- 172.21.15.143:0/1038860075 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f93fc06b340 0x7f93fc06d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.233+0000 7face7e00640 1 --2- 172.21.15.143:0/210879554 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7facc806b2c0 0x7facc806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.233+0000 7f9827e00640 1 --2- 172.21.15.143:0/3706540023 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f980006b2c0 0x7f980006d780 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9818036000 tx=0x7f9818030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.233+0000 7face7e00640 1 -- 172.21.15.143:0/210879554 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7facd80ab500 con 0x7facf00b1250 2024-01-16T15:00:10.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.233+0000 7facefe00640 1 --2- 172.21.15.143:0/210879554 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7facc806b2c0 0x7facc806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.230+0000 7f9079600640 1 -- 172.21.15.143:0/1974852714 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c0723d0 con 0x7f90740e30c0 2024-01-16T15:00:10.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.230+0000 7f9079600640 1 -- 172.21.15.143:0/1974852714 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f907c072570 con 0x7f90740e30c0 2024-01-16T15:00:10.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.230+0000 7f9079600640 1 -- 172.21.15.143:0/1974852714 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c072920 con 0x7f90740e30c0 2024-01-16T15:00:10.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.234+0000 7face7e00640 1 --2- 172.21.15.143:0/210879554 >> [v2:172.21.15.204:6807/3605417837,v1:172.21.15.204:6809/3605417837] conn(0x7facc80734e0 0x7facc8075900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.230+0000 7f9084000640 1 -- 172.21.15.143:0/1974852714 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90740db0f0 con 0x7f90740e30c0 2024-01-16T15:00:10.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.231+0000 7f9084000640 1 -- 172.21.15.143:0/1974852714 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90740db6b0 con 0x7f90740e30c0 2024-01-16T15:00:10.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.232+0000 7f9084000640 1 -- 172.21.15.143:0/1974852714 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9074011f60 con 0x7f90740e30c0 2024-01-16T15:00:10.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.234+0000 7f981fe00640 1 --2- 172.21.15.143:0/3706540023 >> [v2:172.21.15.204:6811/1070255716,v1:172.21.15.204:6812/1070255716] conn(0x7f9800073460 0x7f9800075880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.234+0000 7f9079600640 1 -- 172.21.15.143:0/1974852714 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f907c074940 con 0x7f90740e30c0 2024-01-16T15:00:10.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.234+0000 7facefe00640 1 --2- 172.21.15.143:0/210879554 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7facc806b2c0 0x7facc806d780 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7face0000f60 tx=0x7face0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.234+0000 7face7e00640 1 -- 172.21.15.143:0/210879554 --> [v2:172.21.15.204:6807/3605417837,v1:172.21.15.204:6809/3605417837] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7facc8075fb0 con 0x7facc80734e0 2024-01-16T15:00:10.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.234+0000 7f155aa00640 1 --2- 172.21.15.143:0/4282266046 >> [v2:172.21.15.204:6824/1337220609,v1:172.21.15.204:6825/1337220609] conn(0x7f15340735b0 0x7f15340759d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.234+0000 7face7e00640 1 -- 172.21.15.143:0/210879554 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7facd80ab8b0 con 0x7facf00b1250 2024-01-16T15:00:10.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.235+0000 7facf4e00640 1 --2- 172.21.15.143:0/210879554 >> [v2:172.21.15.204:6807/3605417837,v1:172.21.15.204:6809/3605417837] conn(0x7facc80734e0 0x7facc8075900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.235+0000 7f155b400640 1 --2- 172.21.15.143:0/4282266046 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f153406b390 0x7f153406d850 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f155417c6a0 tx=0x7f154c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.235+0000 7f9079600640 1 --2- 172.21.15.143:0/1974852714 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f905406b2c0 0x7f905406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.235+0000 7f9079600640 1 -- 172.21.15.143:0/1974852714 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f907c104230 con 0x7f90740e30c0 2024-01-16T15:00:10.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.235+0000 7f9429400640 1 --2- 172.21.15.143:0/1038860075 >> [v2:172.21.15.143:6824/3414188012,v1:172.21.15.143:6825/3414188012] conn(0x7f93fc073560 0x7f93fc075980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.235+0000 7f981fe00640 1 --2- 172.21.15.143:0/3706540023 >> [v2:172.21.15.204:6811/1070255716,v1:172.21.15.204:6812/1070255716] conn(0x7f9800073460 0x7f9800075880 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.235+0000 7f9079600640 1 --2- 172.21.15.143:0/1974852714 >> [v2:172.21.15.204:6800/1085000534,v1:172.21.15.204:6801/1085000534] conn(0x7f90540734e0 0x7f9054075900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.235+0000 7f9079600640 1 -- 172.21.15.143:0/1974852714 --> [v2:172.21.15.204:6800/1085000534,v1:172.21.15.204:6801/1085000534] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9054075fb0 con 0x7f90540734e0 2024-01-16T15:00:10.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.235+0000 7f9079600640 1 -- 172.21.15.143:0/1974852714 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f907c1045e0 con 0x7f90740e30c0 2024-01-16T15:00:10.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.235+0000 7f9081600640 1 --2- 172.21.15.143:0/1974852714 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f905406b2c0 0x7f905406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.236+0000 7f9429400640 1 --2- 172.21.15.143:0/1038860075 >> [v2:172.21.15.143:6824/3414188012,v1:172.21.15.143:6825/3414188012] conn(0x7f93fc073560 0x7f93fc075980 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.236+0000 7f9423e00640 1 --2- 172.21.15.143:0/1038860075 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f93fc06b340 0x7f93fc06d800 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f94240b2570 tx=0x7f940c009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.236+0000 7f9080c00640 1 --2- 172.21.15.143:0/1974852714 >> [v2:172.21.15.204:6800/1085000534,v1:172.21.15.204:6801/1085000534] conn(0x7f90540734e0 0x7f9054075900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.236+0000 7facf4e00640 1 --2- 172.21.15.143:0/210879554 >> [v2:172.21.15.204:6807/3605417837,v1:172.21.15.204:6809/3605417837] conn(0x7facc80734e0 0x7facc8075900 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.237+0000 7f9081600640 1 --2- 172.21.15.143:0/1974852714 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f905406b2c0 0x7f905406d780 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f906c000f60 tx=0x7f906c007d50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.237+0000 7f9080c00640 1 --2- 172.21.15.143:0/1974852714 >> [v2:172.21.15.204:6800/1085000534,v1:172.21.15.204:6801/1085000534] conn(0x7f90540734e0 0x7f9054075900 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.237+0000 7f1553400640 1 -- 172.21.15.143:0/4282266046 <== osd.7 v2:172.21.15.204:6824/1337220609 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f1534076080 con 0x7f15340735b0 2024-01-16T15:00:10.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.237+0000 7f9824c00640 1 -- 172.21.15.143:0/3706540023 <== osd.4 v2:172.21.15.204:6811/1070255716 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f9800075f30 con 0x7f9800073460 2024-01-16T15:00:10.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.239+0000 7face7e00640 1 -- 172.21.15.143:0/210879554 <== osd.5 v2:172.21.15.204:6807/3605417837 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7facc8075fb0 con 0x7facc80734e0 2024-01-16T15:00:10.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.239+0000 7f9420c00640 1 -- 172.21.15.143:0/1038860075 <== osd.3 v2:172.21.15.143:6824/3414188012 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f93fc076030 con 0x7f93fc073560 2024-01-16T15:00:10.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.240+0000 7f9079600640 1 -- 172.21.15.143:0/1974852714 <== osd.6 v2:172.21.15.204:6800/1085000534 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f9054075fb0 con 0x7f90540734e0 2024-01-16T15:00:10.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.253+0000 7fcc44800640 1 -- 172.21.15.143:0/328482407 --> [v2:172.21.15.143:6802/1276841168,v1:172.21.15.143:6804/1276841168] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcc3403bf80 con 0x7fcc14073530 2024-01-16T15:00:10.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.255+0000 7fcc3a000640 1 -- 172.21.15.143:0/328482407 <== osd.1 v2:172.21.15.143:6802/1276841168 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcc240095e0 con 0x7fcc14073530 2024-01-16T15:00:10.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.256+0000 7f9a7e800640 1 -- 172.21.15.143:0/2873695655 --> [v2:172.21.15.143:6800/1569293626,v1:172.21.15.143:6801/1569293626] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9a780b03d0 con 0x7f9a500735e0 2024-01-16T15:00:10.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.256+0000 7f9a75600640 1 -- 172.21.15.143:0/2873695655 <== osd.0 v2:172.21.15.143:6800/1569293626 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9a70009d30 con 0x7f9a500735e0 2024-01-16T15:00:10.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.256+0000 7fcc44800640 1 -- 172.21.15.143:0/328482407 >> [v2:172.21.15.143:6802/1276841168,v1:172.21.15.143:6804/1276841168] conn(0x7fcc14073530 msgr2=0x7fcc14075950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.256+0000 7fcc44800640 1 --2- 172.21.15.143:0/328482407 >> [v2:172.21.15.143:6802/1276841168,v1:172.21.15.143:6804/1276841168] conn(0x7fcc14073530 0x7fcc14075950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.257+0000 7fcc44800640 1 -- 172.21.15.143:0/328482407 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fcc1406b390 msgr2=0x7fcc1406d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.257+0000 7fcc44800640 1 --2- 172.21.15.143:0/328482407 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fcc1406b390 0x7fcc1406d850 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcc340085f0 tx=0x7fcc3c0ab000 comp rx=0 tx=0).stop 2024-01-16T15:00:10.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.257+0000 7fcc44800640 1 -- 172.21.15.143:0/328482407 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc340e3080 msgr2=0x7fcc340e4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.257+0000 7fcc44800640 1 --2- 172.21.15.143:0/328482407 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc340e3080 0x7fcc340e4720 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcc3403ecb0 tx=0x7fcc2c00d9c0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.257+0000 7fcc42800640 1 -- 172.21.15.143:0/328482407 reap_dead start 2024-01-16T15:00:10.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.258+0000 7fcc44800640 1 -- 172.21.15.143:0/328482407 shutdown_connections 2024-01-16T15:00:10.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.258+0000 7fcc44800640 1 -- 172.21.15.143:0/328482407 >> 172.21.15.143:0/328482407 conn(0x7fcc340559a0 msgr2=0x7fcc34055d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:10.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.258+0000 7fcc44800640 1 -- 172.21.15.143:0/328482407 shutdown_connections 2024-01-16T15:00:10.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.258+0000 7fcc44800640 1 -- 172.21.15.143:0/328482407 wait complete. 2024-01-16T15:00:10.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.260+0000 7f9a7e800640 1 -- 172.21.15.143:0/2873695655 >> [v2:172.21.15.143:6800/1569293626,v1:172.21.15.143:6801/1569293626] conn(0x7f9a500735e0 msgr2=0x7f9a50075a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.260+0000 7f9a7e800640 1 --2- 172.21.15.143:0/2873695655 >> [v2:172.21.15.143:6800/1569293626,v1:172.21.15.143:6801/1569293626] conn(0x7f9a500735e0 0x7f9a50075a00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.261+0000 7f9a7e800640 1 -- 172.21.15.143:0/2873695655 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f9a5006b3c0 msgr2=0x7f9a5006d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.261+0000 7f9a7e800640 1 --2- 172.21.15.143:0/2873695655 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f9a5006b3c0 0x7f9a5006d880 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9a68036000 tx=0x7f9a68008030 comp rx=0 tx=0).stop 2024-01-16T15:00:10.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.261+0000 7f9a7e800640 1 -- 172.21.15.143:0/2873695655 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a780b1190 msgr2=0x7f9a781524b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.261+0000 7f9a7e800640 1 --2- 172.21.15.143:0/2873695655 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a780b1190 0x7f9a781524b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9a6000c420 tx=0x7f9a6000c8e0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.261+0000 7f9a7de00640 1 -- 172.21.15.143:0/2873695655 reap_dead start 2024-01-16T15:00:10.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.261+0000 7f9a7e800640 1 -- 172.21.15.143:0/2873695655 shutdown_connections 2024-01-16T15:00:10.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.261+0000 7f9a7e800640 1 -- 172.21.15.143:0/2873695655 >> 172.21.15.143:0/2873695655 conn(0x7f9a7808cc90 msgr2=0x7f9a780b54f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:10.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.261+0000 7f9a7e800640 1 -- 172.21.15.143:0/2873695655 shutdown_connections 2024-01-16T15:00:10.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.261+0000 7f9a7e800640 1 -- 172.21.15.143:0/2873695655 wait complete. 2024-01-16T15:00:10.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.262+0000 7ff1a7e00640 1 -- 172.21.15.143:0/1772517210 --> [v2:172.21.15.143:6816/1217670170,v1:172.21.15.143:6817/1217670170] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff178002cb0 con 0x7ff180073540 2024-01-16T15:00:10.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.262+0000 7ff19ea00640 1 -- 172.21.15.143:0/1772517210 <== osd.2 v2:172.21.15.143:6816/1217670170 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff178002cb0 con 0x7ff180073540 2024-01-16T15:00:10.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.263+0000 7ff1a7e00640 1 -- 172.21.15.143:0/1772517210 >> [v2:172.21.15.143:6816/1217670170,v1:172.21.15.143:6817/1217670170] conn(0x7ff180073540 msgr2=0x7ff180075960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.263+0000 7ff1a7e00640 1 --2- 172.21.15.143:0/1772517210 >> [v2:172.21.15.143:6816/1217670170,v1:172.21.15.143:6817/1217670170] conn(0x7ff180073540 0x7ff180075960 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.265+0000 7ff1a7e00640 1 -- 172.21.15.143:0/1772517210 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7ff18006b320 msgr2=0x7ff18006d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.265+0000 7ff1a7e00640 1 --2- 172.21.15.143:0/1772517210 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7ff18006b320 0x7ff18006d7e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff1a00dba90 tx=0x7ff190009040 comp rx=0 tx=0).stop 2024-01-16T15:00:10.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.265+0000 7ff1a7e00640 1 -- 172.21.15.143:0/1772517210 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1a0053c10 msgr2=0x7ff1a003bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.265+0000 7ff1a7e00640 1 --2- 172.21.15.143:0/1772517210 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1a0053c10 0x7ff1a003bad0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff19800c390 tx=0x7ff19800c850 comp rx=0 tx=0).stop 2024-01-16T15:00:10.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.265+0000 7ff1a7400640 1 -- 172.21.15.143:0/1772517210 reap_dead start 2024-01-16T15:00:10.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.265+0000 7ff1a7e00640 1 -- 172.21.15.143:0/1772517210 shutdown_connections 2024-01-16T15:00:10.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.265+0000 7ff1a7e00640 1 -- 172.21.15.143:0/1772517210 >> 172.21.15.143:0/1772517210 conn(0x7ff1a0058eb0 msgr2=0x7ff1a00074d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:10.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.265+0000 7f9084000640 1 -- 172.21.15.143:0/1974852714 --> [v2:172.21.15.204:6800/1085000534,v1:172.21.15.204:6801/1085000534] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f907403bfc0 con 0x7f90540734e0 2024-01-16T15:00:10.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.266+0000 7f9079600640 1 -- 172.21.15.143:0/1974852714 <== osd.6 v2:172.21.15.204:6800/1085000534 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f906403f040 con 0x7f90540734e0 2024-01-16T15:00:10.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.266+0000 7f9429e00640 1 -- 172.21.15.143:0/1038860075 --> [v2:172.21.15.143:6824/3414188012,v1:172.21.15.143:6825/3414188012] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f94240d0630 con 0x7f93fc073560 2024-01-16T15:00:10.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.267+0000 7ff1a7e00640 1 -- 172.21.15.143:0/1772517210 shutdown_connections 2024-01-16T15:00:10.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.267+0000 7ff1a7e00640 1 -- 172.21.15.143:0/1772517210 wait complete. 2024-01-16T15:00:10.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.267+0000 7f9084000640 1 -- 172.21.15.143:0/1974852714 >> [v2:172.21.15.204:6800/1085000534,v1:172.21.15.204:6801/1085000534] conn(0x7f90540734e0 msgr2=0x7f9054075900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.267+0000 7f9084000640 1 --2- 172.21.15.143:0/1974852714 >> [v2:172.21.15.204:6800/1085000534,v1:172.21.15.204:6801/1085000534] conn(0x7f90540734e0 0x7f9054075900 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.267+0000 7f9084000640 1 -- 172.21.15.143:0/1974852714 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f905406b2c0 msgr2=0x7f905406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.267+0000 7f9084000640 1 --2- 172.21.15.143:0/1974852714 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f905406b2c0 0x7f905406d780 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f906c000f60 tx=0x7f906c007d50 comp rx=0 tx=0).stop 2024-01-16T15:00:10.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.267+0000 7f9084000640 1 -- 172.21.15.143:0/1974852714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90740e30c0 msgr2=0x7f90740de180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.267+0000 7f9084000640 1 --2- 172.21.15.143:0/1974852714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90740e30c0 0x7f90740de180 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f907c064180 tx=0x7f907c068010 comp rx=0 tx=0).stop 2024-01-16T15:00:10.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.267+0000 7f9082000640 1 -- 172.21.15.143:0/1974852714 reap_dead start 2024-01-16T15:00:10.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.267+0000 7f9084000640 1 -- 172.21.15.143:0/1974852714 shutdown_connections 2024-01-16T15:00:10.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.267+0000 7f9084000640 1 -- 172.21.15.143:0/1974852714 >> 172.21.15.143:0/1974852714 conn(0x7f90740559e0 msgr2=0x7f9074039790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:10.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.269+0000 7f9084000640 1 -- 172.21.15.143:0/1974852714 shutdown_connections 2024-01-16T15:00:10.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.269+0000 7f9084000640 1 -- 172.21.15.143:0/1974852714 wait complete. 2024-01-16T15:00:10.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.269+0000 7f9420c00640 1 -- 172.21.15.143:0/1038860075 <== osd.3 v2:172.21.15.143:6824/3414188012 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f941c035880 con 0x7f93fc073560 2024-01-16T15:00:10.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.269+0000 7facf5800640 1 -- 172.21.15.143:0/210879554 --> [v2:172.21.15.204:6807/3605417837,v1:172.21.15.204:6809/3605417837] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7facf00ccf40 con 0x7facc80734e0 2024-01-16T15:00:10.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.271+0000 7face7e00640 1 -- 172.21.15.143:0/210879554 <== osd.5 v2:172.21.15.204:6807/3605417837 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7face8009ad0 con 0x7facc80734e0 2024-01-16T15:00:10.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.271+0000 7f982d400640 1 -- 172.21.15.143:0/3706540023 --> [v2:172.21.15.204:6811/1070255716,v1:172.21.15.204:6812/1070255716] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f982804e840 con 0x7f9800073460 2024-01-16T15:00:10.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.272+0000 7f9429e00640 1 -- 172.21.15.143:0/1038860075 >> [v2:172.21.15.143:6824/3414188012,v1:172.21.15.143:6825/3414188012] conn(0x7f93fc073560 msgr2=0x7f93fc075980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.272+0000 7f9429e00640 1 --2- 172.21.15.143:0/1038860075 >> [v2:172.21.15.143:6824/3414188012,v1:172.21.15.143:6825/3414188012] conn(0x7f93fc073560 0x7f93fc075980 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.272+0000 7facf5800640 1 -- 172.21.15.143:0/210879554 >> [v2:172.21.15.204:6807/3605417837,v1:172.21.15.204:6809/3605417837] conn(0x7facc80734e0 msgr2=0x7facc8075900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.272+0000 7facf5800640 1 --2- 172.21.15.143:0/210879554 >> [v2:172.21.15.204:6807/3605417837,v1:172.21.15.204:6809/3605417837] conn(0x7facc80734e0 0x7facc8075900 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.272+0000 7f9429e00640 1 -- 172.21.15.143:0/1038860075 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f93fc06b340 msgr2=0x7f93fc06d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.272+0000 7f9429e00640 1 --2- 172.21.15.143:0/1038860075 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f93fc06b340 0x7f93fc06d800 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f94240b2570 tx=0x7f940c009190 comp rx=0 tx=0).stop 2024-01-16T15:00:10.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.272+0000 7f9429e00640 1 -- 172.21.15.143:0/1038860075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94240ce270 msgr2=0x7f9424160120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.272+0000 7f9429e00640 1 --2- 172.21.15.143:0/1038860075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94240ce270 0x7f9424160120 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f941400c410 tx=0x7f941400c8d0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.272+0000 7f9429400640 1 -- 172.21.15.143:0/1038860075 reap_dead start 2024-01-16T15:00:10.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.272+0000 7f9429e00640 1 -- 172.21.15.143:0/1038860075 shutdown_connections 2024-01-16T15:00:10.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.272+0000 7f9429e00640 1 -- 172.21.15.143:0/1038860075 >> 172.21.15.143:0/1038860075 conn(0x7f9424061340 msgr2=0x7f9424167330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:10.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.272+0000 7f9429e00640 1 -- 172.21.15.143:0/1038860075 shutdown_connections 2024-01-16T15:00:10.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.273+0000 7f9429e00640 1 -- 172.21.15.143:0/1038860075 wait complete. 2024-01-16T15:00:10.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.273+0000 7facf5800640 1 -- 172.21.15.143:0/210879554 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7facc806b2c0 msgr2=0x7facc806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.273+0000 7facf5800640 1 --2- 172.21.15.143:0/210879554 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7facc806b2c0 0x7facc806d780 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7face0000f60 tx=0x7face0009040 comp rx=0 tx=0).stop 2024-01-16T15:00:10.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.273+0000 7facf5800640 1 -- 172.21.15.143:0/210879554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facf00b1250 msgr2=0x7facf00b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.273+0000 7facf5800640 1 --2- 172.21.15.143:0/210879554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facf00b1250 0x7facf00b3630 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7facd803a040 tx=0x7facd8004730 comp rx=0 tx=0).stop 2024-01-16T15:00:10.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.273+0000 7facf4e00640 1 -- 172.21.15.143:0/210879554 reap_dead start 2024-01-16T15:00:10.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.273+0000 7facf5800640 1 -- 172.21.15.143:0/210879554 shutdown_connections 2024-01-16T15:00:10.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.273+0000 7facf5800640 1 -- 172.21.15.143:0/210879554 >> 172.21.15.143:0/210879554 conn(0x7facf00cbb40 msgr2=0x7facf00c1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:10.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.273+0000 7f9824c00640 1 -- 172.21.15.143:0/3706540023 <== osd.4 v2:172.21.15.204:6811/1070255716 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f982804e840 con 0x7f9800073460 2024-01-16T15:00:10.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.274+0000 7f982d400640 1 -- 172.21.15.143:0/3706540023 >> [v2:172.21.15.204:6811/1070255716,v1:172.21.15.204:6812/1070255716] conn(0x7f9800073460 msgr2=0x7f9800075880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.274+0000 7f982d400640 1 --2- 172.21.15.143:0/3706540023 >> [v2:172.21.15.204:6811/1070255716,v1:172.21.15.204:6812/1070255716] conn(0x7f9800073460 0x7f9800075880 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.274+0000 7facf5800640 1 -- 172.21.15.143:0/210879554 shutdown_connections 2024-01-16T15:00:10.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.274+0000 7f982d400640 1 -- 172.21.15.143:0/3706540023 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f980006b2c0 msgr2=0x7f980006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.274+0000 7f982d400640 1 --2- 172.21.15.143:0/3706540023 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f980006b2c0 0x7f980006d780 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9818036000 tx=0x7f9818030000 comp rx=0 tx=0).stop 2024-01-16T15:00:10.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.274+0000 7f982d400640 1 -- 172.21.15.143:0/3706540023 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98280b77c0 msgr2=0x7f98280b7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.274+0000 7f982d400640 1 --2- 172.21.15.143:0/3706540023 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98280b77c0 0x7f98280b7c40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f982000c6d0 tx=0x7f982000cb90 comp rx=0 tx=0).stop 2024-01-16T15:00:10.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.274+0000 7f982ca00640 1 -- 172.21.15.143:0/3706540023 reap_dead start 2024-01-16T15:00:10.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.274+0000 7facf5800640 1 -- 172.21.15.143:0/210879554 wait complete. 2024-01-16T15:00:10.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.274+0000 7f982d400640 1 -- 172.21.15.143:0/3706540023 shutdown_connections 2024-01-16T15:00:10.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.274+0000 7f982d400640 1 -- 172.21.15.143:0/3706540023 >> 172.21.15.143:0/3706540023 conn(0x7f9828061340 msgr2=0x7f98280d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:10.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.274+0000 7f1560a00640 1 -- 172.21.15.143:0/4282266046 --> [v2:172.21.15.204:6824/1337220609,v1:172.21.15.204:6825/1337220609] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f15540572d0 con 0x7f15340735b0 2024-01-16T15:00:10.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.275+0000 7f982d400640 1 -- 172.21.15.143:0/3706540023 shutdown_connections 2024-01-16T15:00:10.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.275+0000 7f982d400640 1 -- 172.21.15.143:0/3706540023 wait complete. 2024-01-16T15:00:10.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.276+0000 7f1553400640 1 -- 172.21.15.143:0/4282266046 <== osd.7 v2:172.21.15.204:6824/1337220609 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1544030b30 con 0x7f15340735b0 2024-01-16T15:00:10.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.277+0000 7f1560a00640 1 -- 172.21.15.143:0/4282266046 >> [v2:172.21.15.204:6824/1337220609,v1:172.21.15.204:6825/1337220609] conn(0x7f15340735b0 msgr2=0x7f15340759d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.277+0000 7f1560a00640 1 --2- 172.21.15.143:0/4282266046 >> [v2:172.21.15.204:6824/1337220609,v1:172.21.15.204:6825/1337220609] conn(0x7f15340735b0 0x7f15340759d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.277+0000 7f1560a00640 1 -- 172.21.15.143:0/4282266046 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f153406b390 msgr2=0x7f153406d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.277+0000 7f1560a00640 1 --2- 172.21.15.143:0/4282266046 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f153406b390 0x7f153406d850 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f155417c6a0 tx=0x7f154c008040 comp rx=0 tx=0).stop 2024-01-16T15:00:10.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.277+0000 7f1560a00640 1 -- 172.21.15.143:0/4282266046 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15540e7330 msgr2=0x7f15540d7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.277+0000 7f1560a00640 1 --2- 172.21.15.143:0/4282266046 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15540e7330 0x7f15540d7300 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f155c0733c0 tx=0x7f155c068010 comp rx=0 tx=0).stop 2024-01-16T15:00:10.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.277+0000 7f155be00640 1 -- 172.21.15.143:0/4282266046 reap_dead start 2024-01-16T15:00:10.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.277+0000 7f1560a00640 1 -- 172.21.15.143:0/4282266046 shutdown_connections 2024-01-16T15:00:10.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.277+0000 7f1560a00640 1 -- 172.21.15.143:0/4282266046 >> 172.21.15.143:0/4282266046 conn(0x7f155403aa00 msgr2=0x7f155403d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:10.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.277+0000 7f1560a00640 1 -- 172.21.15.143:0/4282266046 shutdown_connections 2024-01-16T15:00:10.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.278+0000 7f1560a00640 1 -- 172.21.15.143:0/4282266046 wait complete. 2024-01-16T15:00:10.298 INFO:teuthology.orchestra.run.smithi143.stdout:55834574855 2024-01-16T15:00:10.299 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-01-16T15:00:10.299 INFO:teuthology.orchestra.run.smithi143.stdout:55834574856 2024-01-16T15:00:10.300 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-01-16T15:00:10.300 INFO:teuthology.orchestra.run.smithi143.stdout:55834574855 2024-01-16T15:00:10.301 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-01-16T15:00:10.302 INFO:teuthology.orchestra.run.smithi143.stdout:55834574854 2024-01-16T15:00:10.302 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-01-16T15:00:10.302 INFO:teuthology.orchestra.run.smithi143.stdout:55834574853 2024-01-16T15:00:10.303 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-01-16T15:00:10.303 INFO:teuthology.orchestra.run.smithi143.stdout:55834574856 2024-01-16T15:00:10.304 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-01-16T15:00:10.304 INFO:teuthology.orchestra.run.smithi143.stdout:55834574854 2024-01-16T15:00:10.305 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-01-16T15:00:10.305 INFO:teuthology.orchestra.run.smithi143.stdout:55834574854 2024-01-16T15:00:10.306 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-01-16T15:00:10.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.653+0000 7f47d0a00640 1 Processor -- start 2024-01-16T15:00:10.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.654+0000 7f47d0a00640 1 -- start start 2024-01-16T15:00:10.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.656+0000 7f47d0a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f47cc0b1250 0x7f47cc0ae210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.656+0000 7f47d0a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47cc0cdca0 con 0x7f47cc15d6d0 2024-01-16T15:00:10.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.656+0000 7f47d0a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47cc06c600 con 0x7f47cc15d2f0 2024-01-16T15:00:10.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.656+0000 7f47d0a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47cc06c8e0 con 0x7f47cc0b1250 2024-01-16T15:00:10.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.656+0000 7f47c3e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f47cc15d6d0 0x7f47cc0b0b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54134/0 (socket says 172.21.15.143:54134) 2024-01-16T15:00:10.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.656+0000 7f47c3e00640 1 -- 172.21.15.143:0/38232015 learned_addr learned my addr 172.21.15.143:0/38232015 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:10.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.657+0000 7f9ecb200640 1 Processor -- start 2024-01-16T15:00:10.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.657+0000 7f9ecb200640 1 -- start start 2024-01-16T15:00:10.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.659+0000 7f47cb200640 1 --2- 172.21.15.143:0/38232015 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f47cc0b1250 0x7f47cc0ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.660+0000 7f47cb200640 1 -- 172.21.15.143:0/38232015 >> v1:172.21.15.204:6789/0 conn(0x7f47cc15d2f0 legacy=0x7f47cc0c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.659+0000 7f9ecb200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9ec40af5a0 0x7f9ec40ce9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.660+0000 7f47cb200640 1 -- 172.21.15.143:0/38232015 >> v1:172.21.15.143:6789/0 conn(0x7f47cc15d6d0 legacy=0x7f47cc0b0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.660+0000 7f47cb200640 1 -- 172.21.15.143:0/38232015 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47cc06cbc0 con 0x7f47cc0b1250 2024-01-16T15:00:10.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.659+0000 7f9ecb200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ec40af980 0x7f9ec40cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.659+0000 7f9ecb200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9ec40cd150 0x7f9ec40cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.659+0000 7f9ecb200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ec415dce0 con 0x7f9ec40af5a0 2024-01-16T15:00:10.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.659+0000 7f9ecb200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ec40cf2f0 con 0x7f9ec40cd150 2024-01-16T15:00:10.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.659+0000 7f9ecb200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec40cf5d0 con 0x7f9ec40af980 2024-01-16T15:00:10.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.660+0000 7f9eca800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9ec40cd150 0x7f9ec40cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.660+0000 7f9eca800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9ec40cd150 0x7f9ec40cc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:60620/0 (socket says 172.21.15.143:60620) 2024-01-16T15:00:10.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.660+0000 7f9eca800640 1 -- 172.21.15.143:0/2620083757 learned_addr learned my addr 172.21.15.143:0/2620083757 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:10.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.661+0000 7f9ec9e00640 1 --2- 172.21.15.143:0/2620083757 >> v2:172.21.15.143:3300/0 conn(0x7f9ec40af5a0 0x7f9ec40ce9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.661+0000 7f9ec9400640 1 --2- 172.21.15.143:0/2620083757 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ec40af980 0x7f9ec40cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.661+0000 7f9eca800640 1 -- 172.21.15.143:0/2620083757 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ec40af980 msgr2=0x7f9ec40cbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:10.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.661+0000 7f9eca800640 1 --2- 172.21.15.143:0/2620083757 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ec40af980 0x7f9ec40cbcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.661+0000 7f9eca800640 1 -- 172.21.15.143:0/2620083757 >> v2:172.21.15.143:3300/0 conn(0x7f9ec40af5a0 msgr2=0x7f9ec40ce9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:10.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.662+0000 7f47cb200640 1 --2- 172.21.15.143:0/38232015 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f47cc0b1250 0x7f47cc0ae210 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f47c4009dc0 tx=0x7f47c402bfb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=feb778bf03d67b9f server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.662+0000 7f7a7f400640 1 Processor -- start 2024-01-16T15:00:10.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.662+0000 7f9eca800640 1 --2- 172.21.15.143:0/2620083757 >> v2:172.21.15.143:3300/0 conn(0x7f9ec40af5a0 0x7f9ec40ce9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.662+0000 7f9eca800640 1 -- 172.21.15.143:0/2620083757 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec40cf8b0 con 0x7f9ec40cd150 2024-01-16T15:00:10.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.662+0000 7f9ec9e00640 1 --2- 172.21.15.143:0/2620083757 >> v2:172.21.15.143:3300/0 conn(0x7f9ec40af5a0 0x7f9ec40ce9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T15:00:10.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.662+0000 7f7a7f400640 1 -- start start 2024-01-16T15:00:10.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.664+0000 7f9eca800640 1 --2- 172.21.15.143:0/2620083757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ec40cd150 0x7f9ec40cc200 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9ec40c2dc0 tx=0x7f9eac02b990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fe3b5b0ddcdd32e server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.665+0000 7f7a7f400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7a700e3090 0x7f7a70057d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.665+0000 7f7a7f400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7a700542d0 0x7f7a700068a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.665+0000 7f7a7f400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a700582a0 con 0x7f7a7003e810 2024-01-16T15:00:10.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.665+0000 7f7a7f400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a70006170 con 0x7f7a700542d0 2024-01-16T15:00:10.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.665+0000 7f943a600640 1 Processor -- start 2024-01-16T15:00:10.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.665+0000 7f7a7f400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a70058540 con 0x7f7a700e3090 2024-01-16T15:00:10.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.665+0000 7f943a600640 1 -- start start 2024-01-16T15:00:10.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.665+0000 7f7a7e000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7a700e3090 0x7f7a70057d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.666+0000 7f7a7e000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7a700e3090 0x7f7a70057d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:60626/0 (socket says 172.21.15.143:60626) 2024-01-16T15:00:10.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.666+0000 7f7a7e000640 1 -- 172.21.15.143:0/3671850863 learned_addr learned my addr 172.21.15.143:0/3671850863 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:10.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.666+0000 7f7a7d600640 1 --1- 172.21.15.143:0/3671850863 >> v1:172.21.15.204:6789/0 conn(0x7f7a7003e810 0x7f7a70053bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:33166/0 (socket says 172.21.15.143:33166) 2024-01-16T15:00:10.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.666+0000 7f7a7ea00640 1 --2- 172.21.15.143:0/3671850863 >> v2:172.21.15.143:3300/0 conn(0x7f7a700542d0 0x7f7a700068a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.666+0000 7fbf62600640 1 Processor -- start 2024-01-16T15:00:10.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.666+0000 7f9ec8a00640 1 -- 172.21.15.143:0/2620083757 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac030850 con 0x7f9ec40cd150 2024-01-16T15:00:10.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.666+0000 7f7a7ea00640 1 -- 172.21.15.143:0/3671850863 >> v2:172.21.15.204:3300/0 conn(0x7f7a700e3090 msgr2=0x7f7a70057d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:10.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.667+0000 7f9ec8a00640 1 -- 172.21.15.143:0/2620083757 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eac0309b0 con 0x7f9ec40cd150 2024-01-16T15:00:10.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.667+0000 7f9ec8a00640 1 -- 172.21.15.143:0/2620083757 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac030d40 con 0x7f9ec40cd150 2024-01-16T15:00:10.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.669+0000 7f9ecb200640 1 -- 172.21.15.143:0/2620083757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ec40cd150 msgr2=0x7f9ec40cc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.669+0000 7f9ecb200640 1 --2- 172.21.15.143:0/2620083757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ec40cd150 0x7f9ec40cc200 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9ec40c2dc0 tx=0x7f9eac02b990 comp rx=0 tx=0).stop 2024-01-16T15:00:10.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.667+0000 7f47c9e00640 1 -- 172.21.15.143:0/38232015 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c402cd60 con 0x7f47cc0b1250 2024-01-16T15:00:10.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.668+0000 7f47c9e00640 1 -- 172.21.15.143:0/38232015 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c4034440 con 0x7f47cc0b1250 2024-01-16T15:00:10.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.668+0000 7f47c9e00640 1 -- 172.21.15.143:0/38232015 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c40347d0 con 0x7f47cc0b1250 2024-01-16T15:00:10.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.668+0000 7f47d0a00640 1 -- 172.21.15.143:0/38232015 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f47cc0b1250 msgr2=0x7f47cc0ae210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.668+0000 7f47d0a00640 1 --2- 172.21.15.143:0/38232015 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f47cc0b1250 0x7f47cc0ae210 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f47c4009dc0 tx=0x7f47c402bfb0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.669+0000 7f47d0a00640 1 -- 172.21.15.143:0/38232015 shutdown_connections 2024-01-16T15:00:10.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.669+0000 7f47d0a00640 1 --2- 172.21.15.143:0/38232015 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f47cc0b1250 0x7f47cc0ae210 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.669+0000 7f47d0a00640 1 -- 172.21.15.143:0/38232015 >> 172.21.15.143:0/38232015 conn(0x7f47cc0cbb40 msgr2=0x7f47cc0cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:10.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.667+0000 7f943a600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f943415d490 0x7f94340cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.667+0000 7f943a600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94340b63d0 0x7f94340d3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.667+0000 7f943a600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94340cea30 con 0x7f94340ce270 2024-01-16T15:00:10.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.667+0000 7f943a600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94340d0450 con 0x7f943415d490 2024-01-16T15:00:10.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.667+0000 7f943a600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94340d0bf0 con 0x7f94340b63d0 2024-01-16T15:00:10.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.667+0000 7f7a7ea00640 1 --2- 172.21.15.143:0/3671850863 >> v2:172.21.15.204:3300/0 conn(0x7f7a700e3090 0x7f7a70057d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.667+0000 7f7a7ea00640 1 -- 172.21.15.143:0/3671850863 >> v1:172.21.15.204:6789/0 conn(0x7f7a7003e810 legacy=0x7f7a70053bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.667+0000 7f7a7ea00640 1 -- 172.21.15.143:0/3671850863 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a70058820 con 0x7f7a700542d0 2024-01-16T15:00:10.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.667+0000 7f7a7e000640 1 --2- 172.21.15.143:0/3671850863 >> v2:172.21.15.204:3300/0 conn(0x7f7a700e3090 0x7f7a70057d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T15:00:10.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.668+0000 7f7a7ea00640 1 --2- 172.21.15.143:0/3671850863 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a700542d0 0x7f7a700068a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7a78050140 tx=0x7f7a780a6330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d3821158b44989 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.668+0000 7fcddc000640 1 Processor -- start 2024-01-16T15:00:10.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.668+0000 7fcddc000640 1 -- start start 2024-01-16T15:00:10.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.669+0000 7f9ecb200640 1 -- 172.21.15.143:0/2620083757 shutdown_connections 2024-01-16T15:00:10.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.669+0000 7f9ecb200640 1 --2- 172.21.15.143:0/2620083757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ec40cd150 0x7f9ec40cc200 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.669+0000 7f9ecb200640 1 --2- 172.21.15.143:0/2620083757 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ec40af980 0x7f9ec40cbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.669+0000 7f9ecb200640 1 --2- 172.21.15.143:0/2620083757 >> v2:172.21.15.143:3300/0 conn(0x7f9ec40af5a0 0x7f9ec40ce9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.669+0000 7f9ecb200640 1 -- 172.21.15.143:0/2620083757 >> 172.21.15.143:0/2620083757 conn(0x7f9ec4071b20 msgr2=0x7f9ec4071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:10.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.669+0000 7f9ecb200640 1 -- 172.21.15.143:0/2620083757 shutdown_connections 2024-01-16T15:00:10.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.669+0000 7f9ecb200640 1 -- 172.21.15.143:0/2620083757 wait complete. 2024-01-16T15:00:10.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.669+0000 7f7a7cc00640 1 -- 172.21.15.143:0/3671850863 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a780ae640 con 0x7f7a700542d0 2024-01-16T15:00:10.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.669+0000 7f7a7cc00640 1 -- 172.21.15.143:0/3671850863 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a780ae7e0 con 0x7f7a700542d0 2024-01-16T15:00:10.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.667+0000 7fbf62600640 1 -- start start 2024-01-16T15:00:10.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.669+0000 7f7732000640 1 Processor -- start 2024-01-16T15:00:10.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.669+0000 7fbf62600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbf5c15d500 0x7fbf5c0d0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.669+0000 7fbf62600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf5c0b1190 0x7fbf5c0b1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.669+0000 7fbf62600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf5c15d8e0 con 0x7fbf5c0d30c0 2024-01-16T15:00:10.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.669+0000 7fbf62600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf5c0afea0 con 0x7fbf5c15d500 2024-01-16T15:00:10.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.669+0000 7fbf62600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c0b00f0 con 0x7fbf5c0b1190 2024-01-16T15:00:10.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.670+0000 7f7a7f400640 1 -- 172.21.15.143:0/3671850863 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a700542d0 msgr2=0x7f7a700068a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.668+0000 7f9433e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94340b63d0 0x7f94340d3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.669+0000 7f9439c00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f94340ce270 0x7f9434160e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54146/0 (socket says 172.21.15.143:54146) 2024-01-16T15:00:10.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.669+0000 7f9439c00640 1 -- 172.21.15.143:0/142128868 learned_addr learned my addr 172.21.15.143:0/142128868 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:10.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.671+0000 7f9439200640 1 --2- 172.21.15.143:0/142128868 >> v2:172.21.15.143:3300/0 conn(0x7f943415d490 0x7f94340cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.670+0000 7f7a7f400640 1 --2- 172.21.15.143:0/3671850863 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a700542d0 0x7f7a700068a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7a78050140 tx=0x7f7a780a6330 comp rx=0 tx=0).stop 2024-01-16T15:00:10.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.670+0000 7f7a7f400640 1 -- 172.21.15.143:0/3671850863 shutdown_connections 2024-01-16T15:00:10.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.670+0000 7f7a7f400640 1 --2- 172.21.15.143:0/3671850863 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a700542d0 0x7f7a700068a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.671+0000 7f7a7f400640 1 --2- 172.21.15.143:0/3671850863 >> v2:172.21.15.204:3300/0 conn(0x7f7a700e3090 0x7f7a70057d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.671+0000 7f7a7f400640 1 -- 172.21.15.143:0/3671850863 >> 172.21.15.143:0/3671850863 conn(0x7f7a700559b0 msgr2=0x7f7a70055dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:10.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.670+0000 7fbf61200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbf5c15d500 0x7fbf5c0d0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.671+0000 7fbf61200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbf5c15d500 0x7fbf5c0d0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42622/0 (socket says 172.21.15.143:42622) 2024-01-16T15:00:10.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.671+0000 7fbf61200640 1 -- 172.21.15.143:0/2279221523 learned_addr learned my addr 172.21.15.143:0/2279221523 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:10.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.670+0000 7fcddc000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcdd40cc470 0x7fcdd40abfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.670+0000 7fcddc000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fcdd40ce640 0x7fcdd40b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.670+0000 7fcddc000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcdd40cf450 0x7fcdd40cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.670+0000 7fcddc000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdd40cfe20 con 0x7fcdd40cc470 2024-01-16T15:00:10.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.670+0000 7fcddc000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdd406c560 con 0x7fcdd40ce640 2024-01-16T15:00:10.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.670+0000 7fcddc000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd40ce270 con 0x7fcdd40cf450 2024-01-16T15:00:10.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.670+0000 7f7732000640 1 -- start start 2024-01-16T15:00:10.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.671+0000 7fcdd8c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fcdd40ce640 0x7fcdd40b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.671+0000 7f9ecb200640 1 Processor -- start 2024-01-16T15:00:10.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.671+0000 7f47d0a00640 1 -- 172.21.15.143:0/38232015 shutdown_connections 2024-01-16T15:00:10.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.671+0000 7f47d0a00640 1 -- 172.21.15.143:0/38232015 wait complete. 2024-01-16T15:00:10.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.672+0000 7f7a7f400640 1 -- 172.21.15.143:0/3671850863 shutdown_connections 2024-01-16T15:00:10.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.671+0000 7fcdd8c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fcdd40ce640 0x7fcdd40b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:60632/0 (socket says 172.21.15.143:60632) 2024-01-16T15:00:10.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.671+0000 7fcdd8c00640 1 -- 172.21.15.143:0/2842064249 learned_addr learned my addr 172.21.15.143:0/2842064249 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:10.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.671+0000 7fcdda000640 1 --2- 172.21.15.143:0/2842064249 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcdd40cf450 0x7fcdd40cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.671+0000 7fbf61c00640 1 --2- 172.21.15.143:0/2279221523 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf5c0b1190 0x7fbf5c0b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.672+0000 7f9ecb200640 1 -- start start 2024-01-16T15:00:10.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.672+0000 7f9433e00640 1 -- 172.21.15.143:0/142128868 >> v2:172.21.15.143:3300/0 conn(0x7f943415d490 msgr2=0x7f94340cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:10.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.672+0000 7f9433e00640 1 --2- 172.21.15.143:0/142128868 >> v2:172.21.15.143:3300/0 conn(0x7f943415d490 0x7f94340cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.672+0000 7f9433e00640 1 -- 172.21.15.143:0/142128868 >> v1:172.21.15.143:6789/0 conn(0x7f94340ce270 legacy=0x7f9434160e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.672+0000 7f9433e00640 1 -- 172.21.15.143:0/142128868 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94340d2260 con 0x7f94340b63d0 2024-01-16T15:00:10.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.672+0000 7f7a7f400640 1 -- 172.21.15.143:0/3671850863 wait complete. 2024-01-16T15:00:10.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.672+0000 7fcdda000640 1 -- 172.21.15.143:0/2842064249 >> v2:172.21.15.204:3300/0 conn(0x7fcdd40ce640 msgr2=0x7fcdd40b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:10.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.672+0000 7fbf61200640 1 -- 172.21.15.143:0/2279221523 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf5c0b1190 msgr2=0x7fbf5c0b1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:10.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.672+0000 7fcdda000640 1 --2- 172.21.15.143:0/2842064249 >> v2:172.21.15.204:3300/0 conn(0x7fcdd40ce640 0x7fcdd40b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.672+0000 7fcdda000640 1 -- 172.21.15.143:0/2842064249 >> v2:172.21.15.143:3300/0 conn(0x7fcdd40cc470 msgr2=0x7fcdd40abfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:10.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.672+0000 7f7732000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f772c06d9c0 0x7f772c0d6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.672+0000 7f7732000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f772c0bc900 0x7f772c06dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.672+0000 7f7732000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f772c0d87c0 con 0x7f772c161ee0 2024-01-16T15:00:10.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.672+0000 7f7732000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f772c06e2e0 con 0x7f772c06d9c0 2024-01-16T15:00:10.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.672+0000 7f7732000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f772c166090 con 0x7f772c0bc900 2024-01-16T15:00:10.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.672+0000 7fcdda000640 1 --2- 172.21.15.143:0/2842064249 >> v2:172.21.15.143:3300/0 conn(0x7fcdd40cc470 0x7fcdd40abfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.672+0000 7fcdda000640 1 -- 172.21.15.143:0/2842064249 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd40b4200 con 0x7fcdd40cf450 2024-01-16T15:00:10.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.673+0000 7f772b400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f772c06d9c0 0x7f772c0d6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.673+0000 7f772be00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f772c161ee0 0x7f772c162370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54152/0 (socket says 172.21.15.143:54152) 2024-01-16T15:00:10.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.673+0000 7f772aa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f772c0bc900 0x7f772c06dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.673+0000 7f772be00640 1 -- 172.21.15.143:0/887616563 learned_addr learned my addr 172.21.15.143:0/887616563 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:10.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.674+0000 7f9433e00640 1 --2- 172.21.15.143:0/142128868 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94340b63d0 0x7f94340d3230 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f941c0097a0 tx=0x7f941c02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c628e1d0b3625385 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.674+0000 7f9ecb200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ec40af5a0 0x7f9ec40b4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.674+0000 7f9ecb200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ec40cd150 0x7f9ec41f81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.674+0000 7f9ecb200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ec40c63b0 0x7f9ec40c6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.674+0000 7f9ecb200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec40c6d80 con 0x7f9ec40cd150 2024-01-16T15:00:10.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.674+0000 7f9ecb200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec40b4e40 con 0x7f9ec40af5a0 2024-01-16T15:00:10.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.674+0000 7f9ecb200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec41f8700 con 0x7f9ec40c63b0 2024-01-16T15:00:10.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.674+0000 7f47d0a00640 1 Processor -- start 2024-01-16T15:00:10.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.674+0000 7f772b400640 1 -- 172.21.15.143:0/887616563 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f772c0bc900 msgr2=0x7f772c06dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:10.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.674+0000 7f772b400640 1 --2- 172.21.15.143:0/887616563 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f772c0bc900 0x7f772c06dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.674+0000 7f772b400640 1 -- 172.21.15.143:0/887616563 >> v1:172.21.15.143:6789/0 conn(0x7f772c161ee0 legacy=0x7f772c162370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.674+0000 7f47d0a00640 1 -- start start 2024-01-16T15:00:10.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.675+0000 7f772b400640 1 -- 172.21.15.143:0/887616563 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f772c166370 con 0x7f772c06d9c0 2024-01-16T15:00:10.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.675+0000 7f9ec9e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ec40af5a0 0x7f9ec40b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.675+0000 7f9ec9400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ec40cd150 0x7f9ec41f81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.672+0000 7fbf61200640 1 --2- 172.21.15.143:0/2279221523 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf5c0b1190 0x7fbf5c0b1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.675+0000 7fbf61200640 1 -- 172.21.15.143:0/2279221523 >> v1:172.21.15.143:6789/0 conn(0x7fbf5c0d30c0 legacy=0x7fbf5c0d3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.675+0000 7fbf5b400640 1 -- 172.21.15.143:0/2279221523 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3665631661 0 0) 0x7fbf44001110 con 0x7fbf5c0d30c0 2024-01-16T15:00:10.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.675+0000 7fbf61200640 1 -- 172.21.15.143:0/2279221523 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5c0b03d0 con 0x7fbf5c15d500 2024-01-16T15:00:10.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.675+0000 7fbf61c00640 1 --2- 172.21.15.143:0/2279221523 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf5c0b1190 0x7fbf5c0b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T15:00:10.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.675+0000 7f9ec9400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ec40cd150 0x7f9ec41f81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42658/0 (socket says 172.21.15.143:42658) 2024-01-16T15:00:10.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.675+0000 7f9ec9400640 1 -- 172.21.15.143:0/255855892 learned_addr learned my addr 172.21.15.143:0/255855892 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:10.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.676+0000 7fcdda000640 1 --2- 172.21.15.143:0/2842064249 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcdd40cf450 0x7fcdd40cf8e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcdd40ae200 tx=0x7fcdcc02b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed2917ff952c73c2 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.676+0000 7f47d0a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47cc0b1250 0x7f47cc0b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.676+0000 7f772b400640 1 --2- 172.21.15.143:0/887616563 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f772c06d9c0 0x7f772c0d6e50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f77140097a0 tx=0x7f771402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d2774a0de17e229 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.676+0000 7f9eca800640 1 --2- 172.21.15.143:0/255855892 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ec40c63b0 0x7f9ec40c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.677+0000 7f772a000640 1 -- 172.21.15.143:0/887616563 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7714031780 con 0x7f772c06d9c0 2024-01-16T15:00:10.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.677+0000 7f47d0a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47cc15d2f0 0x7f47cc1546e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.677+0000 7f772a000640 1 -- 172.21.15.143:0/887616563 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77140318e0 con 0x7f772c06d9c0 2024-01-16T15:00:10.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.677+0000 7f772a000640 1 -- 172.21.15.143:0/887616563 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7714031c70 con 0x7f772c06d9c0 2024-01-16T15:00:10.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.677+0000 7f9ec9400640 1 -- 172.21.15.143:0/255855892 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ec40c63b0 msgr2=0x7f9ec40c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.677+0000 7f9ec9400640 1 --2- 172.21.15.143:0/255855892 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ec40c63b0 0x7f9ec40c6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.677+0000 7f9ec9400640 1 -- 172.21.15.143:0/255855892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ec40af5a0 msgr2=0x7f9ec40b4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.677+0000 7f9ec9400640 1 --2- 172.21.15.143:0/255855892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ec40af5a0 0x7f9ec40b4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.677+0000 7f47d0a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f47cc15d6d0 0x7f47cc0999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.677+0000 7f7a7f400640 1 Processor -- start 2024-01-16T15:00:10.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.677+0000 7fbf61200640 1 --2- 172.21.15.143:0/2279221523 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf5c15d500 0x7fbf5c0d0de0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c009870 tx=0x7fbf4c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a83b5c608170cf26 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.677+0000 7f9ec9400640 1 -- 172.21.15.143:0/255855892 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ebc009a70 con 0x7f9ec40cd150 2024-01-16T15:00:10.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.677+0000 7f7a7f400640 1 -- start start 2024-01-16T15:00:10.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.677+0000 7f9eca800640 1 --2- 172.21.15.143:0/255855892 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ec40c63b0 0x7f9ec40c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:10.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.677+0000 7f47d0a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47cc0b4b30 con 0x7f47cc0b1250 2024-01-16T15:00:10.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.677+0000 7fbf5b400640 1 -- 172.21.15.143:0/2279221523 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c031640 con 0x7fbf5c15d500 2024-01-16T15:00:10.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.677+0000 7f47cb200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f47cc15d6d0 0x7f47cc0999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.677+0000 7fbf5b400640 1 -- 172.21.15.143:0/2279221523 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4c0317a0 con 0x7fbf5c15d500 2024-01-16T15:00:10.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.677+0000 7fbf5b400640 1 -- 172.21.15.143:0/2279221523 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c031b30 con 0x7fbf5c15d500 2024-01-16T15:00:10.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.678+0000 7f47cb200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f47cc15d6d0 0x7f47cc0999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:57558/0 (socket says 172.21.15.143:57558) 2024-01-16T15:00:10.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.678+0000 7f47cb200640 1 -- 172.21.15.143:0/2650453809 learned_addr learned my addr 172.21.15.143:0/2650453809 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:10.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.678+0000 7f47ca800640 1 --2- 172.21.15.143:0/2650453809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47cc0b1250 0x7f47cc0b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.678+0000 7f47d0a00640 1 -- 172.21.15.143:0/2650453809 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47cc0b4340 con 0x7f47cc15d2f0 2024-01-16T15:00:10.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.678+0000 7f47d0a00640 1 -- 172.21.15.143:0/2650453809 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47cc154c20 con 0x7f47cc15d6d0 2024-01-16T15:00:10.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.678+0000 7f9ec9400640 1 --2- 172.21.15.143:0/255855892 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ec40cd150 0x7f9ec41f81c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9eb400c820 tx=0x7f9eb400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.678+0000 7f7732000640 1 -- 172.21.15.143:0/887616563 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f772c06d9c0 msgr2=0x7f772c0d6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.678+0000 7f7732000640 1 --2- 172.21.15.143:0/887616563 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f772c06d9c0 0x7f772c0d6e50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f77140097a0 tx=0x7f771402b820 comp rx=0 tx=0).stop 2024-01-16T15:00:10.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.678+0000 7f47c3e00640 1 --2- 172.21.15.143:0/2650453809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47cc15d2f0 0x7f47cc1546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.678+0000 7f7732000640 1 -- 172.21.15.143:0/887616563 shutdown_connections 2024-01-16T15:00:10.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.678+0000 7f7732000640 1 --2- 172.21.15.143:0/887616563 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f772c0bc900 0x7f772c06dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.678+0000 7f7732000640 1 --2- 172.21.15.143:0/887616563 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f772c06d9c0 0x7f772c0d6e50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.678+0000 7f7732000640 1 -- 172.21.15.143:0/887616563 >> 172.21.15.143:0/887616563 conn(0x7f772c0d7840 msgr2=0x7f772c0d7c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:10.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.679+0000 7fbf62600640 1 -- 172.21.15.143:0/2279221523 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf5c15d500 msgr2=0x7fbf5c0d0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.679+0000 7fbf62600640 1 --2- 172.21.15.143:0/2279221523 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf5c15d500 0x7fbf5c0d0de0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c009870 tx=0x7fbf4c02b8f0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.678+0000 7fcdd3e00640 1 -- 172.21.15.143:0/2842064249 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc0317e0 con 0x7fcdd40cf450 2024-01-16T15:00:10.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.679+0000 7fcdd3e00640 1 -- 172.21.15.143:0/2842064249 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdcc031940 con 0x7fcdd40cf450 2024-01-16T15:00:10.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.679+0000 7fcdd3e00640 1 -- 172.21.15.143:0/2842064249 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc031d10 con 0x7fcdd40cf450 2024-01-16T15:00:10.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.678+0000 7f9433400640 1 -- 172.21.15.143:0/142128868 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c02d040 con 0x7f94340b63d0 2024-01-16T15:00:10.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.678+0000 7f9433400640 1 -- 172.21.15.143:0/142128868 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f941c031810 con 0x7f94340b63d0 2024-01-16T15:00:10.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.678+0000 7f9433400640 1 -- 172.21.15.143:0/142128868 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c031ba0 con 0x7f94340b63d0 2024-01-16T15:00:10.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.679+0000 7fbf62600640 1 -- 172.21.15.143:0/2279221523 shutdown_connections 2024-01-16T15:00:10.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.679+0000 7fbf62600640 1 --2- 172.21.15.143:0/2279221523 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf5c0b1190 0x7fbf5c0b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.679+0000 7fbf62600640 1 --2- 172.21.15.143:0/2279221523 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf5c15d500 0x7fbf5c0d0de0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.679+0000 7fbf62600640 1 -- 172.21.15.143:0/2279221523 >> 172.21.15.143:0/2279221523 conn(0x7fbf5c08cc90 msgr2=0x7fbf5c09c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:10.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.679+0000 7f47ca800640 1 -- 172.21.15.143:0/2650453809 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f47cc15d6d0 msgr2=0x7f47cc0999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.679+0000 7f7732000640 1 -- 172.21.15.143:0/887616563 shutdown_connections 2024-01-16T15:00:10.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.679+0000 7f7732000640 1 -- 172.21.15.143:0/887616563 wait complete. 2024-01-16T15:00:10.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.679+0000 7fbf62600640 1 -- 172.21.15.143:0/2279221523 shutdown_connections 2024-01-16T15:00:10.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.679+0000 7f47ca800640 1 --2- 172.21.15.143:0/2650453809 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f47cc15d6d0 0x7f47cc0999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.679+0000 7f47ca800640 1 -- 172.21.15.143:0/2650453809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47cc15d2f0 msgr2=0x7f47cc1546e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.679+0000 7fbf62600640 1 -- 172.21.15.143:0/2279221523 wait complete. 2024-01-16T15:00:10.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.679+0000 7fcddc000640 1 -- 172.21.15.143:0/2842064249 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcdd40cf450 msgr2=0x7fcdd40cf8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.679+0000 7fcddc000640 1 --2- 172.21.15.143:0/2842064249 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcdd40cf450 0x7fcdd40cf8e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcdd40ae200 tx=0x7fcdcc02b880 comp rx=0 tx=0).stop 2024-01-16T15:00:10.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.680+0000 7f7a7f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a7003e810 0x7f7a700ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.680+0000 7f7a7f400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a700542d0 0x7f7a700d8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.680+0000 7fcddc000640 1 -- 172.21.15.143:0/2842064249 shutdown_connections 2024-01-16T15:00:10.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.680+0000 7f7a7f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a700e3090 0x7f7a700d9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.680+0000 7f7a7f400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a700d9970 con 0x7f7a700e3090 2024-01-16T15:00:10.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.680+0000 7f7a7f400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a700d9c50 con 0x7f7a7003e810 2024-01-16T15:00:10.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.680+0000 7f7a7f400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a700d9f30 con 0x7f7a700542d0 2024-01-16T15:00:10.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.680+0000 7fcddc000640 1 --2- 172.21.15.143:0/2842064249 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcdd40cf450 0x7fcdd40cf8e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.680+0000 7fcddc000640 1 --2- 172.21.15.143:0/2842064249 >> v2:172.21.15.204:3300/0 conn(0x7fcdd40ce640 0x7fcdd40b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.680+0000 7fcddc000640 1 --2- 172.21.15.143:0/2842064249 >> v2:172.21.15.143:3300/0 conn(0x7fcdd40cc470 0x7fcdd40abfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.680+0000 7fcddc000640 1 -- 172.21.15.143:0/2842064249 >> 172.21.15.143:0/2842064249 conn(0x7fcdd406a920 msgr2=0x7fcdd406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:10.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.680+0000 7fcddc000640 1 -- 172.21.15.143:0/2842064249 shutdown_connections 2024-01-16T15:00:10.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.680+0000 7f9ec2000640 1 -- 172.21.15.143:0/255855892 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4004220 con 0x7f9ec40cd150 2024-01-16T15:00:10.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.680+0000 7f943a600640 1 -- 172.21.15.143:0/142128868 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94340b63d0 msgr2=0x7f94340d3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.680+0000 7f943a600640 1 --2- 172.21.15.143:0/142128868 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94340b63d0 0x7f94340d3230 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f941c0097a0 tx=0x7f941c02b8b0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.680+0000 7f9ec2000640 1 -- 172.21.15.143:0/255855892 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb40043a0 con 0x7f9ec40cd150 2024-01-16T15:00:10.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.680+0000 7f9ec2000640 1 -- 172.21.15.143:0/255855892 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4007a70 con 0x7f9ec40cd150 2024-01-16T15:00:10.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.681+0000 7f943a600640 1 -- 172.21.15.143:0/142128868 shutdown_connections 2024-01-16T15:00:10.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.681+0000 7f943a600640 1 --2- 172.21.15.143:0/142128868 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94340b63d0 0x7f94340d3230 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.682+0000 7f943a600640 1 --2- 172.21.15.143:0/142128868 >> v2:172.21.15.143:3300/0 conn(0x7f943415d490 0x7f94340cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.682+0000 7f943a600640 1 -- 172.21.15.143:0/142128868 >> 172.21.15.143:0/142128868 conn(0x7f9434061340 msgr2=0x7f9434061750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:10.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.682+0000 7f943a600640 1 -- 172.21.15.143:0/142128868 shutdown_connections 2024-01-16T15:00:10.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.682+0000 7f47ca800640 1 --2- 172.21.15.143:0/2650453809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47cc15d2f0 0x7f47cc1546e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.682+0000 7f47ca800640 1 -- 172.21.15.143:0/2650453809 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c4009a70 con 0x7f47cc0b1250 2024-01-16T15:00:10.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.683+0000 7f47c3e00640 1 --2- 172.21.15.143:0/2650453809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47cc15d2f0 0x7f47cc1546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T15:00:10.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.683+0000 7f47ca800640 1 --2- 172.21.15.143:0/2650453809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47cc0b1250 0x7f47cc0b3e00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00c390 tx=0x7f47bc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.682+0000 7fbf62600640 1 Processor -- start 2024-01-16T15:00:10.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.684+0000 7fbf62600640 1 -- start start 2024-01-16T15:00:10.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.681+0000 7f9ecb200640 1 -- 172.21.15.143:0/255855892 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eac0094d0 con 0x7f9ec40cd150 2024-01-16T15:00:10.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.681+0000 7f9ecb200640 1 -- 172.21.15.143:0/255855892 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ec40c7a50 con 0x7f9ec40cd150 2024-01-16T15:00:10.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.683+0000 7f9ecb200640 1 -- 172.21.15.143:0/255855892 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ec404e840 con 0x7f9ec40cd150 2024-01-16T15:00:10.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.680+0000 7f7a7ea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a700e3090 0x7f7a700d9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.680+0000 7f7a7ea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a700e3090 0x7f7a700d9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42686/0 (socket says 172.21.15.143:42686) 2024-01-16T15:00:10.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.680+0000 7f7a7ea00640 1 -- 172.21.15.143:0/2887041013 learned_addr learned my addr 172.21.15.143:0/2887041013 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:10.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.680+0000 7f7a7d600640 1 --2- 172.21.15.143:0/2887041013 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a700542d0 0x7f7a700d8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.681+0000 7f7a7ea00640 1 -- 172.21.15.143:0/2887041013 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a700542d0 msgr2=0x7f7a700d8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.681+0000 7f7a7ea00640 1 --2- 172.21.15.143:0/2887041013 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a700542d0 0x7f7a700d8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.681+0000 7f7a7ea00640 1 -- 172.21.15.143:0/2887041013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a7003e810 msgr2=0x7f7a700ca370 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-16T15:00:10.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.681+0000 7f7a7ea00640 1 --2- 172.21.15.143:0/2887041013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a7003e810 0x7f7a700ca370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.681+0000 7f7a7ea00640 1 -- 172.21.15.143:0/2887041013 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a680094d0 con 0x7f7a700e3090 2024-01-16T15:00:10.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.681+0000 7f7a7ea00640 1 --2- 172.21.15.143:0/2887041013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a700e3090 0x7f7a700d9340 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7a78050140 tx=0x7f7a7806a3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.681+0000 7f7a76000640 1 -- 172.21.15.143:0/2887041013 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a780ae640 con 0x7f7a700e3090 2024-01-16T15:00:10.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.682+0000 7f7732000640 1 Processor -- start 2024-01-16T15:00:10.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.682+0000 7f7732000640 1 -- start start 2024-01-16T15:00:10.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.682+0000 7f943a600640 1 -- 172.21.15.143:0/142128868 wait complete. 2024-01-16T15:00:10.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.681+0000 7f7a76000640 1 -- 172.21.15.143:0/2887041013 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a780b7400 con 0x7f7a700e3090 2024-01-16T15:00:10.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.681+0000 7f7a76000640 1 -- 172.21.15.143:0/2887041013 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a780b7770 con 0x7f7a700e3090 2024-01-16T15:00:10.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.682+0000 7f7a7f400640 1 -- 172.21.15.143:0/2887041013 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a7804f730 con 0x7f7a700e3090 2024-01-16T15:00:10.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.682+0000 7f7a7f400640 1 -- 172.21.15.143:0/2887041013 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a700da4f0 con 0x7f7a700e3090 2024-01-16T15:00:10.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.684+0000 7f7732000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f772c06d9c0 0x7f772c163770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.683+0000 7f7a7f400640 1 -- 172.21.15.143:0/2887041013 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a700da6e0 con 0x7f7a700e3090 2024-01-16T15:00:10.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.684+0000 7f7732000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f772c0bc900 0x7f772c0b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.684+0000 7f7732000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f772c161ee0 0x7f772c0b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.684+0000 7f7732000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772c0b3910 con 0x7f772c161ee0 2024-01-16T15:00:10.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.684+0000 7f7732000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772c0b3ef0 con 0x7f772c06d9c0 2024-01-16T15:00:10.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.684+0000 7f7732000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f772c15c770 con 0x7f772c0bc900 2024-01-16T15:00:10.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.684+0000 7f943a600640 1 Processor -- start 2024-01-16T15:00:10.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.682+0000 7fcddc000640 1 -- 172.21.15.143:0/2842064249 wait complete. 2024-01-16T15:00:10.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.684+0000 7fcddc000640 1 Processor -- start 2024-01-16T15:00:10.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.685+0000 7f943a600640 1 -- start start 2024-01-16T15:00:10.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.685+0000 7f772aa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f772c0bc900 0x7f772c0b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.685+0000 7f47c2a00640 1 -- 172.21.15.143:0/2650453809 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc0052d0 con 0x7f47cc0b1250 2024-01-16T15:00:10.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.686+0000 7fcddc000640 1 -- start start 2024-01-16T15:00:10.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.685+0000 7f772aa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f772c0bc900 0x7f772c0b2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:57584/0 (socket says 172.21.15.143:57584) 2024-01-16T15:00:10.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.685+0000 7f772aa00640 1 -- 172.21.15.143:0/3205738899 learned_addr learned my addr 172.21.15.143:0/3205738899 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:10.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.686+0000 7fbf62600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf5c0b1190 0x7fbf5c0b88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.686+0000 7fbf62600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf5c0d30c0 0x7fbf5c0b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.686+0000 7f7a76000640 1 -- 172.21.15.143:0/2887041013 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f7a780b5040 con 0x7f7a700e3090 2024-01-16T15:00:10.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.686+0000 7fbf62600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf5c15d500 0x7fbf5c0cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.686+0000 7fbf62600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c0cd550 con 0x7fbf5c15d500 2024-01-16T15:00:10.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.686+0000 7fbf62600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c0b2e10 con 0x7fbf5c0b1190 2024-01-16T15:00:10.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.686+0000 7fbf62600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c0b30f0 con 0x7fbf5c0d30c0 2024-01-16T15:00:10.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.686+0000 7fbf61200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf5c0b1190 0x7fbf5c0b88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.687+0000 7fbf61200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf5c0b1190 0x7fbf5c0b88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:60662/0 (socket says 172.21.15.143:60662) 2024-01-16T15:00:10.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.687+0000 7fbf61200640 1 -- 172.21.15.143:0/1517492274 learned_addr learned my addr 172.21.15.143:0/1517492274 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:10.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.686+0000 7f47d0a00640 1 -- 172.21.15.143:0/2650453809 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47cc0b4570 con 0x7f47cc0b1250 2024-01-16T15:00:10.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.686+0000 7f47d0a00640 1 -- 172.21.15.143:0/2650453809 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47cc0c7340 con 0x7f47cc0b1250 2024-01-16T15:00:10.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.687+0000 7fbf5be00640 1 --2- 172.21.15.143:0/1517492274 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf5c0d30c0 0x7fbf5c0b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.687+0000 7f7a76000640 1 --2- 172.21.15.143:0/2887041013 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f7a5007c2d0 0x7f7a5007e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.686+0000 7f772aa00640 1 -- 172.21.15.143:0/3205738899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f772c06d9c0 msgr2=0x7f772c163770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-16T15:00:10.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.686+0000 7f772aa00640 1 --2- 172.21.15.143:0/3205738899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f772c06d9c0 0x7f772c163770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.686+0000 7f772aa00640 1 -- 172.21.15.143:0/3205738899 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f772c161ee0 msgr2=0x7f772c0b33d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-16T15:00:10.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.686+0000 7f772be00640 1 --2- 172.21.15.143:0/3205738899 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f772c161ee0 0x7f772c0b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.687+0000 7f772aa00640 1 --2- 172.21.15.143:0/3205738899 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f772c161ee0 0x7f772c0b33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.687+0000 7f772aa00640 1 -- 172.21.15.143:0/3205738899 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7714009400 con 0x7f772c0bc900 2024-01-16T15:00:10.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.687+0000 7f772be00640 1 --2- 172.21.15.143:0/3205738899 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f772c161ee0 0x7f772c0b33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:10.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.687+0000 7f47c2a00640 1 -- 172.21.15.143:0/2650453809 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47bc005430 con 0x7f47cc0b1250 2024-01-16T15:00:10.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.688+0000 7f47c2a00640 1 -- 172.21.15.143:0/2650453809 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc003830 con 0x7f47cc0b1250 2024-01-16T15:00:10.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.687+0000 7fbf61200640 1 -- 172.21.15.143:0/1517492274 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf5c0d30c0 msgr2=0x7fbf5c0b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.688+0000 7fbf61200640 1 --2- 172.21.15.143:0/1517492274 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf5c0d30c0 0x7fbf5c0b5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.688+0000 7fbf61200640 1 -- 172.21.15.143:0/1517492274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf5c15d500 msgr2=0x7fbf5c0cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.688+0000 7fbf61200640 1 --2- 172.21.15.143:0/1517492274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf5c15d500 0x7fbf5c0cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.688+0000 7fbf61200640 1 -- 172.21.15.143:0/1517492274 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf4c0094d0 con 0x7fbf5c0b1190 2024-01-16T15:00:10.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.687+0000 7f7a76000640 1 -- 172.21.15.143:0/2887041013 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7a78125f20 con 0x7f7a700e3090 2024-01-16T15:00:10.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.688+0000 7f943a600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94340ce270 0x7f943415eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.688+0000 7f7a7e000640 1 --2- 172.21.15.143:0/2887041013 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f7a5007c2d0 0x7f7a5007e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.688+0000 7fbf61200640 1 --2- 172.21.15.143:0/1517492274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf5c0b1190 0x7fbf5c0b88b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c009870 tx=0x7fbf4c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.687+0000 7fcddc000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcdd40cc470 0x7fcdd4153610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.687+0000 7fcddc000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdd40ce640 0x7fcdd4153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.687+0000 7fcddc000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcdd40cf450 0x7fcdd4159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.688+0000 7fcddc000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd4153b50 con 0x7fcdd40cf450 2024-01-16T15:00:10.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.688+0000 7fcddc000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd415a870 con 0x7fcdd40ce640 2024-01-16T15:00:10.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.688+0000 7fcddc000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd415ab50 con 0x7fcdd40cc470 2024-01-16T15:00:10.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.688+0000 7f4beec00640 1 Processor -- start 2024-01-16T15:00:10.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.689+0000 7fbf58c00640 1 -- 172.21.15.143:0/1517492274 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c03ac60 con 0x7fbf5c0b1190 2024-01-16T15:00:10.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.689+0000 7fbf62600640 1 -- 172.21.15.143:0/1517492274 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf5c0b33d0 con 0x7fbf5c0b1190 2024-01-16T15:00:10.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.689+0000 7fcdda000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcdd40cf450 0x7fcdd4159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.689+0000 7fbf62600640 1 -- 172.21.15.143:0/1517492274 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf5c0b3990 con 0x7fbf5c0b1190 2024-01-16T15:00:10.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.688+0000 7f772aa00640 1 --2- 172.21.15.143:0/3205738899 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f772c0bc900 0x7f772c0b2e90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f771c00e9b0 tx=0x7f771c00ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.689+0000 7f7723400640 1 -- 172.21.15.143:0/3205738899 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771c009670 con 0x7f772c0bc900 2024-01-16T15:00:10.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.689+0000 7fcdd9600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcdd40cc470 0x7fcdd4153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.689+0000 7fcdd9600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcdd40cc470 0x7fcdd4153610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:57598/0 (socket says 172.21.15.143:57598) 2024-01-16T15:00:10.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.689+0000 7fcdd9600640 1 -- 172.21.15.143:0/495103016 learned_addr learned my addr 172.21.15.143:0/495103016 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:10.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.689+0000 7f4beec00640 1 -- start start 2024-01-16T15:00:10.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.689+0000 7fbf58c00640 1 -- 172.21.15.143:0/1517492274 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4c031e60 con 0x7fbf5c0b1190 2024-01-16T15:00:10.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.689+0000 7fbf58c00640 1 -- 172.21.15.143:0/1517492274 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c04d450 con 0x7fbf5c0b1190 2024-01-16T15:00:10.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.690+0000 7f7a7e000640 1 --2- 172.21.15.143:0/2887041013 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f7a5007c2d0 0x7f7a5007e790 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7a68009b90 tx=0x7f7a68009240 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.690+0000 7f7723400640 1 -- 172.21.15.143:0/3205738899 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f771c00c9a0 con 0x7f772c0bc900 2024-01-16T15:00:10.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.690+0000 7f4beec00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4be80cfb60 0x7f4be80abfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.690+0000 7f943a600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f943415d490 0x7f94340b7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.690+0000 7f4beec00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4be80ce640 0x7f4be80b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.690+0000 7f4beec00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4be80ce270 con 0x7f4be80cc470 2024-01-16T15:00:10.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.690+0000 7f4beec00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4be806c560 con 0x7f4be80cfb60 2024-01-16T15:00:10.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.690+0000 7f4beec00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be806d0a0 con 0x7f4be80ce640 2024-01-16T15:00:10.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.691+0000 7fcdd9600640 1 -- 172.21.15.143:0/495103016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdd40ce640 msgr2=0x7fcdd4153cc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-16T15:00:10.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.691+0000 7f943a600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94340ccf60 0x7f9434154120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.691+0000 7fcdd9600640 1 --2- 172.21.15.143:0/495103016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdd40ce640 0x7fcdd4153cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.691+0000 7fcdd9600640 1 -- 172.21.15.143:0/495103016 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcdd40cf450 msgr2=0x7fcdd4159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.691+0000 7fcdd9600640 1 --2- 172.21.15.143:0/495103016 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcdd40cf450 0x7fcdd4159130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.691+0000 7fcdd9600640 1 -- 172.21.15.143:0/495103016 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdcc009400 con 0x7fcdd40cc470 2024-01-16T15:00:10.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.691+0000 7f9433e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f943415d490 0x7f94340b7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.691+0000 7f4becc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4be80ce640 0x7f4be80b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.691+0000 7f4becc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4be80ce640 0x7f4be80b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:57600/0 (socket says 172.21.15.143:57600) 2024-01-16T15:00:10.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.691+0000 7f4becc00640 1 -- 172.21.15.143:0/2666894939 learned_addr learned my addr 172.21.15.143:0/2666894939 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:10.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.691+0000 7f47d0a00640 1 -- 172.21.15.143:0/2650453809 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f479c005250 con 0x7f47cc0b1250 2024-01-16T15:00:10.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.691+0000 7fcdda000640 1 --2- 172.21.15.143:0/495103016 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcdd40cf450 0x7fcdd4159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T15:00:10.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.691+0000 7f4bdfe00640 1 --2- 172.21.15.143:0/2666894939 >> v2:172.21.15.143:3300/0 conn(0x7f4be80cfb60 0x7f4be80abfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.691+0000 7f9433e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f943415d490 0x7f94340b7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:57612/0 (socket says 172.21.15.143:57612) 2024-01-16T15:00:10.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.690+0000 7f7732000640 1 -- 172.21.15.143:0/3205738899 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f772c0b4190 con 0x7f772c0bc900 2024-01-16T15:00:10.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.690+0000 7f7732000640 1 -- 172.21.15.143:0/3205738899 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f772c0b4520 con 0x7f772c0bc900 2024-01-16T15:00:10.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.690+0000 7f7723400640 1 -- 172.21.15.143:0/3205738899 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771c00cd50 con 0x7f772c0bc900 2024-01-16T15:00:10.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.692+0000 7fcdd9600640 1 --2- 172.21.15.143:0/495103016 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcdd40cc470 0x7fcdd4153610 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc00b4a0 tx=0x7fcdbc00b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.692+0000 7f7732000640 1 -- 172.21.15.143:0/3205738899 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76fc005250 con 0x7f772c0bc900 2024-01-16T15:00:10.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.692+0000 7f4bdfe00640 1 -- 172.21.15.143:0/2666894939 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4be80ce640 msgr2=0x7f4be80b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:10.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.692+0000 7f4bdfe00640 1 --2- 172.21.15.143:0/2666894939 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4be80ce640 0x7f4be80b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.692+0000 7f4bdfe00640 1 -- 172.21.15.143:0/2666894939 >> v1:172.21.15.204:6789/0 conn(0x7f4be80cc470 legacy=0x7f4be80cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:10.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.692+0000 7f4bdfe00640 1 -- 172.21.15.143:0/2666894939 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be80b4200 con 0x7f4be80cfb60 2024-01-16T15:00:10.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.692+0000 7f9433e00640 1 -- 172.21.15.143:0/3836902175 learned_addr learned my addr 172.21.15.143:0/3836902175 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:10.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.692+0000 7f9439c00640 1 --2- 172.21.15.143:0/3836902175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94340ccf60 0x7f9434154120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.692+0000 7f9ec2000640 1 -- 172.21.15.143:0/255855892 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f9eb4007bd0 con 0x7f9ec40cd150 2024-01-16T15:00:10.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.692+0000 7f9439200640 1 --2- 172.21.15.143:0/3836902175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94340ce270 0x7f943415eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.692+0000 7f943a600640 1 -- 172.21.15.143:0/3836902175 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94340cbec0 con 0x7f94340ccf60 2024-01-16T15:00:10.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.693+0000 7f943a600640 1 -- 172.21.15.143:0/3836902175 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94340cc1a0 con 0x7f94340ce270 2024-01-16T15:00:10.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.693+0000 7f943a600640 1 -- 172.21.15.143:0/3836902175 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94340cc480 con 0x7f943415d490 2024-01-16T15:00:10.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.693+0000 7fcdd1600640 1 -- 172.21.15.143:0/495103016 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc010610 con 0x7fcdd40cc470 2024-01-16T15:00:10.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.693+0000 7fcddc000640 1 -- 172.21.15.143:0/495103016 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdd415ae30 con 0x7fcdd40cc470 2024-01-16T15:00:10.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.692+0000 7fbf62600640 1 -- 172.21.15.143:0/1517492274 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf2c005250 con 0x7fbf5c0b1190 2024-01-16T15:00:10.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.692+0000 7fbf58c00640 1 -- 172.21.15.143:0/1517492274 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fbf4c02f050 con 0x7fbf5c0b1190 2024-01-16T15:00:10.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.693+0000 7f4bdfe00640 1 --2- 172.21.15.143:0/2666894939 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4be80cfb60 0x7f4be80abfc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4bd00097a0 tx=0x7f4bd002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4787e186b5b05459 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.693+0000 7fcddc000640 1 -- 172.21.15.143:0/495103016 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdd415b3f0 con 0x7fcdd40cc470 2024-01-16T15:00:10.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.694+0000 7fbf58c00640 1 --2- 172.21.15.143:0/1517492274 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbf3406b2f0 0x7fbf3406d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.694+0000 7f9439200640 1 -- 172.21.15.143:0/3836902175 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f943415d490 msgr2=0x7f94340b7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.694+0000 7f9439200640 1 --2- 172.21.15.143:0/3836902175 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f943415d490 0x7f94340b7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.694+0000 7f9439200640 1 -- 172.21.15.143:0/3836902175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94340ccf60 msgr2=0x7f9434154120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.694+0000 7fbf58c00640 1 -- 172.21.15.143:0/1517492274 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbf4c0ab9f0 con 0x7fbf5c0b1190 2024-01-16T15:00:10.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.694+0000 7f9ec2000640 1 --2- 172.21.15.143:0/255855892 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f9e9c06b310 0x7f9e9c06d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.694+0000 7f9ec2000640 1 -- 172.21.15.143:0/255855892 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9eb4089de0 con 0x7f9ec40cd150 2024-01-16T15:00:10.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.694+0000 7f9439200640 1 --2- 172.21.15.143:0/3836902175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94340ccf60 0x7f9434154120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.694+0000 7f9ec9e00640 1 --2- 172.21.15.143:0/255855892 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f9e9c06b310 0x7f9e9c06d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.694+0000 7f9439200640 1 -- 172.21.15.143:0/3836902175 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f941c009400 con 0x7f94340ce270 2024-01-16T15:00:10.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.695+0000 7f9439c00640 1 --2- 172.21.15.143:0/3836902175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94340ccf60 0x7f9434154120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T15:00:10.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.695+0000 7fbf5be00640 1 --2- 172.21.15.143:0/1517492274 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbf3406b2f0 0x7fbf3406d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.695+0000 7f9ec9e00640 1 --2- 172.21.15.143:0/255855892 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f9e9c06b310 0x7f9e9c06d7d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc009fd0 tx=0x7f9ebc00d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.695+0000 7f9439200640 1 --2- 172.21.15.143:0/3836902175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94340ce270 0x7f943415eda0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f942400e870 tx=0x7f942400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.696+0000 7f9ec2000640 1 -- 172.21.15.143:0/255855892 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9eb4054ce0 con 0x7f9ec40cd150 2024-01-16T15:00:10.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.696+0000 7fcddc000640 1 -- 172.21.15.143:0/495103016 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdd4067470 con 0x7fcdd40cc470 2024-01-16T15:00:10.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.696+0000 7f7723400640 1 -- 172.21.15.143:0/3205738899 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f771c013470 con 0x7f772c0bc900 2024-01-16T15:00:10.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.696+0000 7fbf5be00640 1 --2- 172.21.15.143:0/1517492274 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbf3406b2f0 0x7fbf3406d7b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c142f50 tx=0x7fbf44009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.697+0000 7f7723400640 1 --2- 172.21.15.143:0/3205738899 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f770406b2c0 0x7f770406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.697+0000 7f7723400640 1 -- 172.21.15.143:0/3205738899 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f771c08a540 con 0x7f772c0bc900 2024-01-16T15:00:10.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.697+0000 7f772b400640 1 --2- 172.21.15.143:0/3205738899 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f770406b2c0 0x7f770406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.698+0000 7f772b400640 1 --2- 172.21.15.143:0/3205738899 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f770406b2c0 0x7f770406d780 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7714036000 tx=0x7f7714008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.700+0000 7f7a76000640 1 -- 172.21.15.143:0/2887041013 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7a780f0cf0 con 0x7f7a700e3090 2024-01-16T15:00:10.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.700+0000 7f4be7400640 1 -- 172.21.15.143:0/2666894939 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0031780 con 0x7f4be80cfb60 2024-01-16T15:00:10.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.700+0000 7f4be7400640 1 -- 172.21.15.143:0/2666894939 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd00318e0 con 0x7f4be80cfb60 2024-01-16T15:00:10.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.700+0000 7f4be7400640 1 -- 172.21.15.143:0/2666894939 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0031c70 con 0x7f4be80cfb60 2024-01-16T15:00:10.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.701+0000 7f4beec00640 1 -- 172.21.15.143:0/2666894939 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4be80cfb60 msgr2=0x7f4be80abfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.701+0000 7f4beec00640 1 --2- 172.21.15.143:0/2666894939 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4be80cfb60 0x7f4be80abfc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4bd00097a0 tx=0x7f4bd002b820 comp rx=0 tx=0).stop 2024-01-16T15:00:10.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.701+0000 7f4beec00640 1 -- 172.21.15.143:0/2666894939 shutdown_connections 2024-01-16T15:00:10.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.701+0000 7f4beec00640 1 --2- 172.21.15.143:0/2666894939 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4be80ce640 0x7f4be80b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.701+0000 7f4beec00640 1 --2- 172.21.15.143:0/2666894939 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4be80cfb60 0x7f4be80abfc0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.701+0000 7f4beec00640 1 -- 172.21.15.143:0/2666894939 >> 172.21.15.143:0/2666894939 conn(0x7f4be806a920 msgr2=0x7f4be806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:10.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.701+0000 7f4beec00640 1 -- 172.21.15.143:0/2666894939 shutdown_connections 2024-01-16T15:00:10.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.701+0000 7f4beec00640 1 -- 172.21.15.143:0/2666894939 wait complete. 2024-01-16T15:00:10.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.702+0000 7f47c2a00640 1 -- 172.21.15.143:0/2650453809 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f47bc003990 con 0x7f47cc0b1250 2024-01-16T15:00:10.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.703+0000 7f47c2a00640 1 --2- 172.21.15.143:0/2650453809 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f47a406b2c0 0x7f47a406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.703+0000 7f47c2a00640 1 -- 172.21.15.143:0/2650453809 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f47bc089510 con 0x7f47cc0b1250 2024-01-16T15:00:10.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.704+0000 7f47c3e00640 1 --2- 172.21.15.143:0/2650453809 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f47a406b2c0 0x7f47a406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.704+0000 7f4beec00640 1 Processor -- start 2024-01-16T15:00:10.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.704+0000 7f4beec00640 1 -- start start 2024-01-16T15:00:10.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.705+0000 7f47c3e00640 1 --2- 172.21.15.143:0/2650453809 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f47a406b2c0 0x7f47a406d780 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f47cc072960 tx=0x7f47b4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.705+0000 7f4beec00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4be80cc470 0x7f4be814f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.705+0000 7f4beec00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4be80ce640 0x7f4be814f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.705+0000 7f4beec00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4be80cfb60 0x7f4be8152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.705+0000 7f4beec00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be81500e0 con 0x7f4be80ce640 2024-01-16T15:00:10.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.705+0000 7f4beec00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be8164cb0 con 0x7f4be80cc470 2024-01-16T15:00:10.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.705+0000 7f4beec00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be8150380 con 0x7f4be80cfb60 2024-01-16T15:00:10.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.706+0000 7f4bdfe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4be80ce640 0x7f4be814f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.706+0000 7f4bdfe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4be80ce640 0x7f4be814f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42760/0 (socket says 172.21.15.143:42760) 2024-01-16T15:00:10.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.706+0000 7f4bdfe00640 1 -- 172.21.15.143:0/808704810 learned_addr learned my addr 172.21.15.143:0/808704810 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:10.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.706+0000 7f4be7e00640 1 --2- 172.21.15.143:0/808704810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4be80cc470 0x7f4be814f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.706+0000 7f4becc00640 1 --2- 172.21.15.143:0/808704810 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4be80cfb60 0x7f4be8152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.707+0000 7f47c2a00640 1 -- 172.21.15.143:0/2650453809 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f47bc054410 con 0x7f47cc0b1250 2024-01-16T15:00:10.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.707+0000 7f4bdfe00640 1 -- 172.21.15.143:0/808704810 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4be80cfb60 msgr2=0x7f4be8152340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.707+0000 7f4bdfe00640 1 --2- 172.21.15.143:0/808704810 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4be80cfb60 0x7f4be8152340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.707+0000 7f4bdfe00640 1 -- 172.21.15.143:0/808704810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4be80cc470 msgr2=0x7f4be814f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:10.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.707+0000 7f4bdfe00640 1 --2- 172.21.15.143:0/808704810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4be80cc470 0x7f4be814f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:10.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.707+0000 7f4bdfe00640 1 -- 172.21.15.143:0/808704810 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd0009400 con 0x7f4be80ce640 2024-01-16T15:00:10.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.707+0000 7f4bdfe00640 1 --2- 172.21.15.143:0/808704810 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4be80ce640 0x7f4be814f930 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0034040 tx=0x7f4bd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.708+0000 7f4be4c00640 1 -- 172.21.15.143:0/808704810 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0032620 con 0x7f4be80ce640 2024-01-16T15:00:10.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.708+0000 7f4be4c00640 1 -- 172.21.15.143:0/808704810 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd00327c0 con 0x7f4be80ce640 2024-01-16T15:00:10.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.708+0000 7f4beec00640 1 -- 172.21.15.143:0/808704810 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be8150660 con 0x7f4be80ce640 2024-01-16T15:00:10.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.708+0000 7f4be4c00640 1 -- 172.21.15.143:0/808704810 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0032b70 con 0x7f4be80ce640 2024-01-16T15:00:10.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.708+0000 7f4beec00640 1 -- 172.21.15.143:0/808704810 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be8152880 con 0x7f4be80ce640 2024-01-16T15:00:10.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.709+0000 7f4beec00640 1 -- 172.21.15.143:0/808704810 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be8150880 con 0x7f4be80ce640 2024-01-16T15:00:10.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.712+0000 7f4be4c00640 1 -- 172.21.15.143:0/808704810 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f4bd0032cd0 con 0x7f4be80ce640 2024-01-16T15:00:10.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.712+0000 7f9430c00640 1 -- 172.21.15.143:0/3836902175 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942400ee60 con 0x7f94340ce270 2024-01-16T15:00:10.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.712+0000 7f9430c00640 1 -- 172.21.15.143:0/3836902175 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942400c940 con 0x7f94340ce270 2024-01-16T15:00:10.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.712+0000 7fcdd1600640 1 -- 172.21.15.143:0/495103016 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdbc010770 con 0x7fcdd40cc470 2024-01-16T15:00:10.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.712+0000 7fcdd1600640 1 -- 172.21.15.143:0/495103016 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc010ae0 con 0x7fcdd40cc470 2024-01-16T15:00:10.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.712+0000 7f943a600640 1 -- 172.21.15.143:0/3836902175 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94340cc760 con 0x7f94340ce270 2024-01-16T15:00:10.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.712+0000 7f943a600640 1 -- 172.21.15.143:0/3836902175 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f943415f2e0 con 0x7f94340ce270 2024-01-16T15:00:10.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.712+0000 7f7723400640 1 -- 172.21.15.143:0/3205738899 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f771c055420 con 0x7f772c0bc900 2024-01-16T15:00:10.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.713+0000 7f9430c00640 1 -- 172.21.15.143:0/3836902175 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942400ccd0 con 0x7f94340ce270 2024-01-16T15:00:10.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.713+0000 7f4be4c00640 1 --2- 172.21.15.143:0/808704810 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f4bc006b310 0x7f4bc006d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.713+0000 7f4be4c00640 1 -- 172.21.15.143:0/808704810 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4bd007acb0 con 0x7f4be80ce640 2024-01-16T15:00:10.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.714+0000 7f4be7e00640 1 --2- 172.21.15.143:0/808704810 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f4bc006b310 0x7f4bc006d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.714+0000 7f943a600640 1 -- 172.21.15.143:0/3836902175 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9404005250 con 0x7f94340ce270 2024-01-16T15:00:10.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.715+0000 7fbf58c00640 1 -- 172.21.15.143:0/1517492274 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbf4c076970 con 0x7fbf5c0b1190 2024-01-16T15:00:10.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.715+0000 7f4be7e00640 1 --2- 172.21.15.143:0/808704810 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f4bc006b310 0x7f4bc006d7d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4bd80017f0 tx=0x7f4bd8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.717+0000 7fcdd1600640 1 -- 172.21.15.143:0/495103016 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fcdbc010c40 con 0x7fcdd40cc470 2024-01-16T15:00:10.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.717+0000 7f9430c00640 1 -- 172.21.15.143:0/3836902175 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f9424013470 con 0x7f94340ce270 2024-01-16T15:00:10.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.718+0000 7f9430c00640 1 --2- 172.21.15.143:0/3836902175 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f940c06b390 0x7f940c06d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.718+0000 7fcdd1600640 1 --2- 172.21.15.143:0/495103016 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fcdac06b2f0 0x7fcdac06d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:10.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.719+0000 7f4be4c00640 1 -- 172.21.15.143:0/808704810 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4bd0077780 con 0x7f4be80ce640 2024-01-16T15:00:10.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.719+0000 7fcdd1600640 1 -- 172.21.15.143:0/495103016 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcdbc089bb0 con 0x7fcdd40cc470 2024-01-16T15:00:10.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.719+0000 7f9430c00640 1 -- 172.21.15.143:0/3836902175 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9424089b90 con 0x7f94340ce270 2024-01-16T15:00:10.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.719+0000 7f9433e00640 1 --2- 172.21.15.143:0/3836902175 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f940c06b390 0x7f940c06d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.719+0000 7fcdd8c00640 1 --2- 172.21.15.143:0/495103016 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fcdac06b2f0 0x7fcdac06d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:10.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.720+0000 7fcdd8c00640 1 --2- 172.21.15.143:0/495103016 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fcdac06b2f0 0x7fcdac06d7b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcdd40b0710 tx=0x7fcdc400b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.721+0000 7f9433e00640 1 --2- 172.21.15.143:0/3836902175 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f940c06b390 0x7f940c06d850 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f941c02b8f0 tx=0x7f941c03c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:10.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.726+0000 7fcdd1600640 1 -- 172.21.15.143:0/495103016 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcdbc054ab0 con 0x7fcdd40cc470 2024-01-16T15:00:10.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:10.731+0000 7f9430c00640 1 -- 172.21.15.143:0/3836902175 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9424054a90 con 0x7f94340ce270 2024-01-16T15:00:11.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.120+0000 7f9ecb200640 1 -- 172.21.15.143:0/255855892 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9ec40b5260 con 0x7f9ec40cd150 2024-01-16T15:00:11.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.120+0000 7f7a7f400640 1 -- 172.21.15.143:0/2887041013 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f7a700da6e0 con 0x7f7a700e3090 2024-01-16T15:00:11.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.121+0000 7f9ec2000640 1 -- 172.21.15.143:0/255855892 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9eb401e020 con 0x7f9ec40cd150 2024-01-16T15:00:11.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.121+0000 7f7a76000640 1 -- 172.21.15.143:0/2887041013 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7a780b3020 con 0x7f7a700e3090 2024-01-16T15:00:11.102 INFO:teuthology.orchestra.run.smithi143.stdout:55834574853 2024-01-16T15:00:11.103 INFO:teuthology.orchestra.run.smithi143.stdout:55834574854 2024-01-16T15:00:11.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.124+0000 7f7a6f400640 1 -- 172.21.15.143:0/2887041013 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f7a5007c2d0 msgr2=0x7f7a5007e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:11.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.124+0000 7f7a6f400640 1 --2- 172.21.15.143:0/2887041013 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f7a5007c2d0 0x7f7a5007e790 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7a68009b90 tx=0x7f7a68009240 comp rx=0 tx=0).stop 2024-01-16T15:00:11.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.124+0000 7f7a6f400640 1 -- 172.21.15.143:0/2887041013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a700e3090 msgr2=0x7f7a700d9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:11.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.124+0000 7f7a6f400640 1 --2- 172.21.15.143:0/2887041013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a700e3090 0x7f7a700d9340 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7a78050140 tx=0x7f7a7806a3b0 comp rx=0 tx=0).stop 2024-01-16T15:00:11.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.125+0000 7f7a6f400640 1 -- 172.21.15.143:0/2887041013 shutdown_connections 2024-01-16T15:00:11.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.125+0000 7f7a6f400640 1 --2- 172.21.15.143:0/2887041013 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f7a5007c2d0 0x7f7a5007e790 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:11.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.125+0000 7f7a6f400640 1 --2- 172.21.15.143:0/2887041013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a700e3090 0x7f7a700d9340 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:11.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.125+0000 7f7a6f400640 1 --2- 172.21.15.143:0/2887041013 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a700542d0 0x7f7a700d8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:11.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.125+0000 7f7a6f400640 1 --2- 172.21.15.143:0/2887041013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a7003e810 0x7f7a700ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:11.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.125+0000 7f7a6f400640 1 -- 172.21.15.143:0/2887041013 >> 172.21.15.143:0/2887041013 conn(0x7f7a700559b0 msgr2=0x7f7a70055d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:11.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.125+0000 7f9ecb200640 1 -- 172.21.15.143:0/255855892 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f9e9c06b310 msgr2=0x7f9e9c06d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:11.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.125+0000 7f9ecb200640 1 --2- 172.21.15.143:0/255855892 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f9e9c06b310 0x7f9e9c06d7d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc009fd0 tx=0x7f9ebc00d040 comp rx=0 tx=0).stop 2024-01-16T15:00:11.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.125+0000 7f7a6f400640 1 -- 172.21.15.143:0/2887041013 shutdown_connections 2024-01-16T15:00:11.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.125+0000 7f9ecb200640 1 -- 172.21.15.143:0/255855892 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ec40cd150 msgr2=0x7f9ec41f81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:11.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.125+0000 7f9ecb200640 1 --2- 172.21.15.143:0/255855892 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ec40cd150 0x7f9ec41f81c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9eb400c820 tx=0x7f9eb400cce0 comp rx=0 tx=0).stop 2024-01-16T15:00:11.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.126+0000 7f7a6f400640 1 -- 172.21.15.143:0/2887041013 wait complete. 2024-01-16T15:00:11.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.126+0000 7f9ecb200640 1 -- 172.21.15.143:0/255855892 shutdown_connections 2024-01-16T15:00:11.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.126+0000 7f9ecb200640 1 --2- 172.21.15.143:0/255855892 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f9e9c06b310 0x7f9e9c06d7d0 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc009fd0 tx=0x7f9ebc00d040 comp rx=0 tx=0).stop 2024-01-16T15:00:11.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.126+0000 7f9ecb200640 1 --2- 172.21.15.143:0/255855892 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ec40c63b0 0x7f9ec40c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:11.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.126+0000 7f9ecb200640 1 --2- 172.21.15.143:0/255855892 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ec40cd150 0x7f9ec41f81c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:11.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.126+0000 7f9ecb200640 1 --2- 172.21.15.143:0/255855892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ec40af5a0 0x7f9ec40b4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:11.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.126+0000 7f9ecb200640 1 -- 172.21.15.143:0/255855892 >> 172.21.15.143:0/255855892 conn(0x7f9ec4071b20 msgr2=0x7f9ec40afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:11.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.127+0000 7f9ecb200640 1 -- 172.21.15.143:0/255855892 shutdown_connections 2024-01-16T15:00:11.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.127+0000 7f9ecb200640 1 -- 172.21.15.143:0/255855892 wait complete. 2024-01-16T15:00:11.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.127+0000 7f47d0a00640 1 -- 172.21.15.143:0/2650453809 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f479c0057d0 con 0x7f47cc0b1250 2024-01-16T15:00:11.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.128+0000 7f47c2a00640 1 -- 172.21.15.143:0/2650453809 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f47bc057a30 con 0x7f47cc0b1250 2024-01-16T15:00:11.110 INFO:teuthology.orchestra.run.smithi143.stdout:55834574855 2024-01-16T15:00:11.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.131+0000 7fbf62600640 1 -- 172.21.15.143:0/1517492274 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fbf2c005500 con 0x7fbf5c0b1190 2024-01-16T15:00:11.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.132+0000 7f47d0a00640 1 -- 172.21.15.143:0/2650453809 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f47a406b2c0 msgr2=0x7f47a406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:11.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.132+0000 7f47d0a00640 1 --2- 172.21.15.143:0/2650453809 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f47a406b2c0 0x7f47a406d780 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f47cc072960 tx=0x7f47b4009040 comp rx=0 tx=0).stop 2024-01-16T15:00:11.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.132+0000 7f47d0a00640 1 -- 172.21.15.143:0/2650453809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47cc0b1250 msgr2=0x7f47cc0b3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:11.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.132+0000 7f47d0a00640 1 --2- 172.21.15.143:0/2650453809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47cc0b1250 0x7f47cc0b3e00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00c390 tx=0x7f47bc00c850 comp rx=0 tx=0).stop 2024-01-16T15:00:11.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.132+0000 7fbf58c00640 1 -- 172.21.15.143:0/1517492274 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbf4c035090 con 0x7fbf5c0b1190 2024-01-16T15:00:11.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.132+0000 7f7732000640 1 -- 172.21.15.143:0/3205738899 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f76fc0057d0 con 0x7f772c0bc900 2024-01-16T15:00:11.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.133+0000 7f47d0a00640 1 -- 172.21.15.143:0/2650453809 shutdown_connections 2024-01-16T15:00:11.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.133+0000 7f47d0a00640 1 --2- 172.21.15.143:0/2650453809 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f47a406b2c0 0x7f47a406d780 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:11.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.133+0000 7f47d0a00640 1 --2- 172.21.15.143:0/2650453809 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f47cc15d6d0 0x7f47cc0999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:11.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.133+0000 7f47d0a00640 1 --2- 172.21.15.143:0/2650453809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47cc15d2f0 0x7f47cc1546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:11.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.133+0000 7f47d0a00640 1 --2- 172.21.15.143:0/2650453809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47cc0b1250 0x7f47cc0b3e00 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:11.115 INFO:teuthology.orchestra.run.smithi143.stdout:55834574852 2024-01-16T15:00:11.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.133+0000 7f47d0a00640 1 -- 172.21.15.143:0/2650453809 >> 172.21.15.143:0/2650453809 conn(0x7f47cc0cbb40 msgr2=0x7f47cc0b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:11.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.134+0000 7f47d0a00640 1 -- 172.21.15.143:0/2650453809 shutdown_connections 2024-01-16T15:00:11.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.134+0000 7f47d0a00640 1 -- 172.21.15.143:0/2650453809 wait complete. 2024-01-16T15:00:11.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.134+0000 7f7723400640 1 -- 172.21.15.143:0/3205738899 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f771c058a40 con 0x7f772c0bc900 2024-01-16T15:00:11.116 INFO:teuthology.orchestra.run.smithi143.stdout:55834574853 2024-01-16T15:00:11.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.136+0000 7f4beec00640 1 -- 172.21.15.143:0/808704810 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f4be8067470 con 0x7f4be80ce640 2024-01-16T15:00:11.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.137+0000 7fbf62600640 1 -- 172.21.15.143:0/1517492274 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbf3406b2f0 msgr2=0x7fbf3406d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:11.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.137+0000 7fbf62600640 1 --2- 172.21.15.143:0/1517492274 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbf3406b2f0 0x7fbf3406d7b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c142f50 tx=0x7fbf44009040 comp rx=0 tx=0).stop 2024-01-16T15:00:11.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.137+0000 7fbf62600640 1 -- 172.21.15.143:0/1517492274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf5c0b1190 msgr2=0x7fbf5c0b88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:11.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.137+0000 7fbf62600640 1 --2- 172.21.15.143:0/1517492274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf5c0b1190 0x7fbf5c0b88b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c009870 tx=0x7fbf4c0046c0 comp rx=0 tx=0).stop 2024-01-16T15:00:11.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.137+0000 7fbf62600640 1 -- 172.21.15.143:0/1517492274 shutdown_connections 2024-01-16T15:00:11.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.137+0000 7fbf62600640 1 --2- 172.21.15.143:0/1517492274 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbf3406b2f0 0x7fbf3406d7b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:11.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.137+0000 7fbf62600640 1 --2- 172.21.15.143:0/1517492274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf5c15d500 0x7fbf5c0cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:11.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.137+0000 7fbf62600640 1 --2- 172.21.15.143:0/1517492274 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf5c0d30c0 0x7fbf5c0b5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:11.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.137+0000 7fbf62600640 1 --2- 172.21.15.143:0/1517492274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf5c0b1190 0x7fbf5c0b88b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:11.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.137+0000 7fbf62600640 1 -- 172.21.15.143:0/1517492274 >> 172.21.15.143:0/1517492274 conn(0x7fbf5c08cc90 msgr2=0x7fbf5c0cdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:11.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.138+0000 7f4be4c00640 1 -- 172.21.15.143:0/808704810 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4bd0049020 con 0x7f4be80ce640 2024-01-16T15:00:11.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.138+0000 7fbf62600640 1 -- 172.21.15.143:0/1517492274 shutdown_connections 2024-01-16T15:00:11.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.138+0000 7f7732000640 1 -- 172.21.15.143:0/3205738899 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f770406b2c0 msgr2=0x7f770406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:11.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.138+0000 7f7732000640 1 --2- 172.21.15.143:0/3205738899 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f770406b2c0 0x7f770406d780 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7714036000 tx=0x7f7714008030 comp rx=0 tx=0).stop 2024-01-16T15:00:11.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.138+0000 7fbf62600640 1 -- 172.21.15.143:0/1517492274 wait complete. 2024-01-16T15:00:11.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.138+0000 7f7732000640 1 -- 172.21.15.143:0/3205738899 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f772c0bc900 msgr2=0x7f772c0b2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:11.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.138+0000 7f7732000640 1 --2- 172.21.15.143:0/3205738899 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f772c0bc900 0x7f772c0b2e90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f771c00e9b0 tx=0x7f771c00ee70 comp rx=0 tx=0).stop 2024-01-16T15:00:11.121 INFO:teuthology.orchestra.run.smithi143.stdout:55834574853 2024-01-16T15:00:11.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.139+0000 7f7732000640 1 -- 172.21.15.143:0/3205738899 shutdown_connections 2024-01-16T15:00:11.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.139+0000 7f7732000640 1 --2- 172.21.15.143:0/3205738899 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f770406b2c0 0x7f770406d780 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:11.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.139+0000 7f7732000640 1 --2- 172.21.15.143:0/3205738899 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f772c161ee0 0x7f772c0b33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:11.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.139+0000 7f7732000640 1 --2- 172.21.15.143:0/3205738899 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f772c0bc900 0x7f772c0b2e90 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:11.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.139+0000 7f7732000640 1 --2- 172.21.15.143:0/3205738899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f772c06d9c0 0x7f772c163770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:11.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.139+0000 7f7732000640 1 -- 172.21.15.143:0/3205738899 >> 172.21.15.143:0/3205738899 conn(0x7f772c0d7840 msgr2=0x7f772c0bbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:11.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.139+0000 7f7732000640 1 -- 172.21.15.143:0/3205738899 shutdown_connections 2024-01-16T15:00:11.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.139+0000 7f7732000640 1 -- 172.21.15.143:0/3205738899 wait complete. 2024-01-16T15:00:11.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.142+0000 7fcddc000640 1 -- 172.21.15.143:0/495103016 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fcdd41665c0 con 0x7fcdd40cc470 2024-01-16T15:00:11.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.142+0000 7f4beec00640 1 -- 172.21.15.143:0/808704810 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f4bc006b310 msgr2=0x7f4bc006d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:11.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.142+0000 7f4beec00640 1 --2- 172.21.15.143:0/808704810 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f4bc006b310 0x7f4bc006d7d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4bd80017f0 tx=0x7f4bd8009040 comp rx=0 tx=0).stop 2024-01-16T15:00:11.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.143+0000 7f4beec00640 1 -- 172.21.15.143:0/808704810 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4be80ce640 msgr2=0x7f4be814f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:11.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.143+0000 7f4beec00640 1 --2- 172.21.15.143:0/808704810 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4be80ce640 0x7f4be814f930 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0034040 tx=0x7f4bd00046c0 comp rx=0 tx=0).stop 2024-01-16T15:00:11.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.143+0000 7f4beec00640 1 -- 172.21.15.143:0/808704810 shutdown_connections 2024-01-16T15:00:11.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.143+0000 7f4beec00640 1 --2- 172.21.15.143:0/808704810 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f4bc006b310 0x7f4bc006d7d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:11.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.143+0000 7f4beec00640 1 --2- 172.21.15.143:0/808704810 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4be80cfb60 0x7f4be8152340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:11.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.143+0000 7f4beec00640 1 --2- 172.21.15.143:0/808704810 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4be80ce640 0x7f4be814f930 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:11.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.143+0000 7f4beec00640 1 --2- 172.21.15.143:0/808704810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4be80cc470 0x7f4be814f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:11.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.143+0000 7f4beec00640 1 -- 172.21.15.143:0/808704810 >> 172.21.15.143:0/808704810 conn(0x7f4be806a920 msgr2=0x7f4be806d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:11.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.143+0000 7fcdd1600640 1 -- 172.21.15.143:0/495103016 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcdbc01e090 con 0x7fcdd40cc470 2024-01-16T15:00:11.126 INFO:teuthology.orchestra.run.smithi143.stdout:55834574855 2024-01-16T15:00:11.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.144+0000 7f4beec00640 1 -- 172.21.15.143:0/808704810 shutdown_connections 2024-01-16T15:00:11.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.144+0000 7f4beec00640 1 -- 172.21.15.143:0/808704810 wait complete. 2024-01-16T15:00:11.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.148+0000 7fcddc000640 1 -- 172.21.15.143:0/495103016 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fcdac06b2f0 msgr2=0x7fcdac06d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:11.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.148+0000 7fcddc000640 1 --2- 172.21.15.143:0/495103016 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fcdac06b2f0 0x7fcdac06d7b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcdd40b0710 tx=0x7fcdc400b040 comp rx=0 tx=0).stop 2024-01-16T15:00:11.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.148+0000 7fcddc000640 1 -- 172.21.15.143:0/495103016 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcdd40cc470 msgr2=0x7fcdd4153610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:11.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.148+0000 7fcddc000640 1 --2- 172.21.15.143:0/495103016 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcdd40cc470 0x7fcdd4153610 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc00b4a0 tx=0x7fcdbc00b960 comp rx=0 tx=0).stop 2024-01-16T15:00:11.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.148+0000 7fcddc000640 1 -- 172.21.15.143:0/495103016 shutdown_connections 2024-01-16T15:00:11.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.148+0000 7fcddc000640 1 --2- 172.21.15.143:0/495103016 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fcdac06b2f0 0x7fcdac06d7b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:11.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.148+0000 7fcddc000640 1 --2- 172.21.15.143:0/495103016 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcdd40cf450 0x7fcdd4159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:11.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.149+0000 7fcddc000640 1 --2- 172.21.15.143:0/495103016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdd40ce640 0x7fcdd4153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:11.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.149+0000 7fcddc000640 1 --2- 172.21.15.143:0/495103016 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcdd40cc470 0x7fcdd4153610 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:11.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.149+0000 7fcddc000640 1 -- 172.21.15.143:0/495103016 >> 172.21.15.143:0/495103016 conn(0x7fcdd406a920 msgr2=0x7fcdd40acd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:11.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.149+0000 7fcddc000640 1 -- 172.21.15.143:0/495103016 shutdown_connections 2024-01-16T15:00:11.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.149+0000 7fcddc000640 1 -- 172.21.15.143:0/495103016 wait complete. 2024-01-16T15:00:11.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.150+0000 7f943a600640 1 -- 172.21.15.143:0/3836902175 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9404005500 con 0x7f94340ce270 2024-01-16T15:00:11.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.151+0000 7f9430c00640 1 -- 172.21.15.143:0/3836902175 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9424017090 con 0x7f94340ce270 2024-01-16T15:00:11.134 INFO:teuthology.orchestra.run.smithi143.stdout:55834574854 2024-01-16T15:00:11.135 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.6 2024-01-16T15:00:11.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.156+0000 7f943a600640 1 -- 172.21.15.143:0/3836902175 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f940c06b390 msgr2=0x7f940c06d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:11.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.156+0000 7f943a600640 1 --2- 172.21.15.143:0/3836902175 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f940c06b390 0x7f940c06d850 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f941c02b8f0 tx=0x7f941c03c040 comp rx=0 tx=0).stop 2024-01-16T15:00:11.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.156+0000 7f943a600640 1 -- 172.21.15.143:0/3836902175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94340ce270 msgr2=0x7f943415eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:11.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.156+0000 7f943a600640 1 --2- 172.21.15.143:0/3836902175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94340ce270 0x7f943415eda0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f942400e870 tx=0x7f942400ed30 comp rx=0 tx=0).stop 2024-01-16T15:00:11.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.156+0000 7f943a600640 1 -- 172.21.15.143:0/3836902175 shutdown_connections 2024-01-16T15:00:11.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.156+0000 7f943a600640 1 --2- 172.21.15.143:0/3836902175 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f940c06b390 0x7f940c06d850 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f941c02b8f0 tx=0x7f941c03c040 comp rx=0 tx=0).stop 2024-01-16T15:00:11.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.156+0000 7f943a600640 1 --2- 172.21.15.143:0/3836902175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94340ccf60 0x7f9434154120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:11.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.156+0000 7f943a600640 1 --2- 172.21.15.143:0/3836902175 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f943415d490 0x7f94340b7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:11.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.156+0000 7f943a600640 1 --2- 172.21.15.143:0/3836902175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94340ce270 0x7f943415eda0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:11.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.156+0000 7f943a600640 1 -- 172.21.15.143:0/3836902175 >> 172.21.15.143:0/3836902175 conn(0x7f9434061340 msgr2=0x7f94340b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:11.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.157+0000 7f943a600640 1 -- 172.21.15.143:0/3836902175 shutdown_connections 2024-01-16T15:00:11.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:11.157+0000 7f943a600640 1 -- 172.21.15.143:0/3836902175 wait complete. 2024-01-16T15:00:11.139 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.0 2024-01-16T15:00:11.148 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.1 2024-01-16T15:00:11.150 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2024-01-16T15:00:11.150 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.3 2024-01-16T15:00:11.153 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.4 2024-01-16T15:00:11.156 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.5 2024-01-16T15:00:11.159 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2024-01-16T15:00:12.136 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-01-16T15:00:12.140 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-01-16T15:00:12.149 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-01-16T15:00:12.150 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-01-16T15:00:12.151 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-01-16T15:00:12.154 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-01-16T15:00:12.158 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-01-16T15:00:12.160 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-01-16T15:00:12.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.424+0000 7f62a9800640 1 Processor -- start 2024-01-16T15:00:12.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.425+0000 7f62a9800640 1 -- start start 2024-01-16T15:00:12.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.427+0000 7f62a9800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f629c0075b0 0x7f629c037c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.427+0000 7f62a9800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f629c052fd0 con 0x7f629c04f290 2024-01-16T15:00:12.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.427+0000 7f62a9800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f629c053270 con 0x7f629c1051e0 2024-01-16T15:00:12.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.427+0000 7f62a9800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f629c0503a0 con 0x7f629c0075b0 2024-01-16T15:00:12.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.428+0000 7f62a2a00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f629c1051e0 0x7f629c054010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:33190/0 (socket says 172.21.15.143:33190) 2024-01-16T15:00:12.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.428+0000 7f62a2a00640 1 -- 172.21.15.143:0/3430035064 learned_addr learned my addr 172.21.15.143:0/3430035064 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:12.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.428+0000 7f62a3400640 1 --2- 172.21.15.143:0/3430035064 >> v2:172.21.15.143:3300/0 conn(0x7f629c0075b0 0x7f629c037c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.429+0000 7f62a3400640 1 -- 172.21.15.143:0/3430035064 >> v1:172.21.15.204:6789/0 conn(0x7f629c1051e0 legacy=0x7f629c054010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.429+0000 7f62a3400640 1 -- 172.21.15.143:0/3430035064 >> v1:172.21.15.143:6789/0 conn(0x7f629c04f290 legacy=0x7f629c006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.429+0000 7f62a3400640 1 -- 172.21.15.143:0/3430035064 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f629c053530 con 0x7f629c0075b0 2024-01-16T15:00:12.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.431+0000 7f2f2f800640 1 Processor -- start 2024-01-16T15:00:12.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.431+0000 7f2f2f800640 1 -- start start 2024-01-16T15:00:12.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.432+0000 7f62a3400640 1 --2- 172.21.15.143:0/3430035064 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f629c0075b0 0x7f629c037c20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f62a4050140 tx=0x7f62a40953a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c87835cb77d1a3 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:12.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.432+0000 7f62a1600640 1 -- 172.21.15.143:0/3430035064 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a409a7d0 con 0x7f629c0075b0 2024-01-16T15:00:12.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.433+0000 7f62a1600640 1 -- 172.21.15.143:0/3430035064 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62a4098050 con 0x7f629c0075b0 2024-01-16T15:00:12.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.433+0000 7f62a1600640 1 -- 172.21.15.143:0/3430035064 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a409ac30 con 0x7f629c0075b0 2024-01-16T15:00:12.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.433+0000 7f2f2f800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2f2814b240 0x7f2f28071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.433+0000 7f2f2f800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f28071950 0x7f2f2806c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.433+0000 7f2f2f800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2f2806ca30 0x7f2f28165430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.434+0000 7f2f2f800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f28071db0 con 0x7f2f2814b240 2024-01-16T15:00:12.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.434+0000 7f2f2ce00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2f2814b240 0x7f2f28071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.434+0000 7f2f2ce00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2f2814b240 0x7f2f28071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42774/0 (socket says 172.21.15.143:42774) 2024-01-16T15:00:12.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.434+0000 7f2f2f800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f2806cec0 con 0x7f2f2806ca30 2024-01-16T15:00:12.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.434+0000 7f2f2f800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f28070830 con 0x7f2f28071950 2024-01-16T15:00:12.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.434+0000 7f2f2ce00640 1 -- 172.21.15.143:0/1540278709 learned_addr learned my addr 172.21.15.143:0/1540278709 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:12.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.435+0000 7f62a9800640 1 -- 172.21.15.143:0/3430035064 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f629c0075b0 msgr2=0x7f629c037c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.435+0000 7f62a9800640 1 --2- 172.21.15.143:0/3430035064 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f629c0075b0 0x7f629c037c20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f62a4050140 tx=0x7f62a40953a0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.435+0000 7f62a9800640 1 -- 172.21.15.143:0/3430035064 shutdown_connections 2024-01-16T15:00:12.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.435+0000 7f62a9800640 1 --2- 172.21.15.143:0/3430035064 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f629c0075b0 0x7f629c037c20 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.435+0000 7f62a9800640 1 -- 172.21.15.143:0/3430035064 >> 172.21.15.143:0/3430035064 conn(0x7f629c009fb0 msgr2=0x7f629c00a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:12.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.434+0000 7f2f2d800640 1 --2- 172.21.15.143:0/1540278709 >> v2:172.21.15.204:3300/0 conn(0x7f2f2806ca30 0x7f2f28165430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.434+0000 7f2f27e00640 1 --2- 172.21.15.143:0/1540278709 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f28071950 0x7f2f2806c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.435+0000 7f2f27e00640 1 -- 172.21.15.143:0/1540278709 >> v2:172.21.15.204:3300/0 conn(0x7f2f2806ca30 msgr2=0x7f2f28165430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:12.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.435+0000 7f2f27e00640 1 --2- 172.21.15.143:0/1540278709 >> v2:172.21.15.204:3300/0 conn(0x7f2f2806ca30 0x7f2f28165430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.435+0000 7f2f27e00640 1 -- 172.21.15.143:0/1540278709 >> v2:172.21.15.143:3300/0 conn(0x7f2f2814b240 msgr2=0x7f2f28071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:12.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.435+0000 7f2f27e00640 1 --2- 172.21.15.143:0/1540278709 >> v2:172.21.15.143:3300/0 conn(0x7f2f2814b240 0x7f2f28071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.435+0000 7f2f27e00640 1 -- 172.21.15.143:0/1540278709 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f28161de0 con 0x7f2f28071950 2024-01-16T15:00:12.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.436+0000 7f2f2ce00640 1 --2- 172.21.15.143:0/1540278709 >> v2:172.21.15.143:3300/0 conn(0x7f2f2814b240 0x7f2f28071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T15:00:12.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.437+0000 7f62a9800640 1 -- 172.21.15.143:0/3430035064 shutdown_connections 2024-01-16T15:00:12.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.437+0000 7f62a9800640 1 -- 172.21.15.143:0/3430035064 wait complete. 2024-01-16T15:00:12.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.439+0000 7f2f27e00640 1 --2- 172.21.15.143:0/1540278709 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f28071950 0x7f2f2806c4f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2f28070470 tx=0x7f2f1802b7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=643645d33d6cd707 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:12.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.439+0000 7f2f27400640 1 -- 172.21.15.143:0/1540278709 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f18031520 con 0x7f2f28071950 2024-01-16T15:00:12.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.439+0000 7f2f27400640 1 -- 172.21.15.143:0/1540278709 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f18031680 con 0x7f2f28071950 2024-01-16T15:00:12.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.439+0000 7f2f27400640 1 -- 172.21.15.143:0/1540278709 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f18031a10 con 0x7f2f28071950 2024-01-16T15:00:12.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.440+0000 7f2f2f800640 1 -- 172.21.15.143:0/1540278709 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f28071950 msgr2=0x7f2f2806c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.440+0000 7f2f2f800640 1 --2- 172.21.15.143:0/1540278709 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f28071950 0x7f2f2806c4f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2f28070470 tx=0x7f2f1802b7c0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.441+0000 7f62a9800640 1 Processor -- start 2024-01-16T15:00:12.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.441+0000 7f2f2f800640 1 -- 172.21.15.143:0/1540278709 shutdown_connections 2024-01-16T15:00:12.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.441+0000 7f62a9800640 1 -- start start 2024-01-16T15:00:12.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.441+0000 7f2f2f800640 1 --2- 172.21.15.143:0/1540278709 >> v2:172.21.15.204:3300/0 conn(0x7f2f2806ca30 0x7f2f28165430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.441+0000 7f2f2f800640 1 --2- 172.21.15.143:0/1540278709 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f28071950 0x7f2f2806c4f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.441+0000 7f2f2f800640 1 --2- 172.21.15.143:0/1540278709 >> v2:172.21.15.143:3300/0 conn(0x7f2f2814b240 0x7f2f28071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.441+0000 7f2f2f800640 1 -- 172.21.15.143:0/1540278709 >> 172.21.15.143:0/1540278709 conn(0x7f2f2806dc30 msgr2=0x7f2f2806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:12.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.442+0000 7f2f2f800640 1 -- 172.21.15.143:0/1540278709 shutdown_connections 2024-01-16T15:00:12.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.443+0000 7f62a9800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f629c0075b0 0x7f629c1a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.443+0000 7f62a9800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f629c04f290 0x7f629c02b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.443+0000 7f62a9800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f629c1051e0 0x7f629c02d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.443+0000 7f62a9800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629c04cbe0 con 0x7f629c1051e0 2024-01-16T15:00:12.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.443+0000 7f62a9800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629c1a1b90 con 0x7f629c04f290 2024-01-16T15:00:12.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.443+0000 7f62a9800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f629c02bd50 con 0x7f629c0075b0 2024-01-16T15:00:12.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.442+0000 7f2f2f800640 1 -- 172.21.15.143:0/1540278709 wait complete. 2024-01-16T15:00:12.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.444+0000 7f62a3400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f629c1051e0 0x7f629c02d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.444+0000 7f62a3400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f629c1051e0 0x7f629c02d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42786/0 (socket says 172.21.15.143:42786) 2024-01-16T15:00:12.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.444+0000 7f62a3400640 1 -- 172.21.15.143:0/2448881880 learned_addr learned my addr 172.21.15.143:0/2448881880 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:12.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.445+0000 7f62a3400640 1 -- 172.21.15.143:0/2448881880 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f629c0075b0 msgr2=0x7f629c1a1650 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-16T15:00:12.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.445+0000 7f62a3400640 1 --2- 172.21.15.143:0/2448881880 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f629c0075b0 0x7f629c1a1650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.445+0000 7f62a3400640 1 -- 172.21.15.143:0/2448881880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f629c04f290 msgr2=0x7f629c02b470 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-16T15:00:12.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.445+0000 7f62a3400640 1 --2- 172.21.15.143:0/2448881880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f629c04f290 0x7f629c02b470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.446+0000 7f62a3400640 1 -- 172.21.15.143:0/2448881880 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a404f730 con 0x7f629c1051e0 2024-01-16T15:00:12.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.446+0000 7f2f2f800640 1 Processor -- start 2024-01-16T15:00:12.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.446+0000 7f62a3400640 1 --2- 172.21.15.143:0/2448881880 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f629c1051e0 0x7f629c02d380 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f62a402cee0 tx=0x7f62a406a3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:12.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.446+0000 7f2f2f800640 1 -- start start 2024-01-16T15:00:12.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.447+0000 7f629aa00640 1 -- 172.21.15.143:0/2448881880 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a409dad0 con 0x7f629c1051e0 2024-01-16T15:00:12.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.447+0000 7f629aa00640 1 -- 172.21.15.143:0/2448881880 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62a409dc30 con 0x7f629c1051e0 2024-01-16T15:00:12.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.447+0000 7f629aa00640 1 -- 172.21.15.143:0/2448881880 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a40b0410 con 0x7f629c1051e0 2024-01-16T15:00:12.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.447+0000 7f62a9800640 1 -- 172.21.15.143:0/2448881880 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f629c02c030 con 0x7f629c1051e0 2024-01-16T15:00:12.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.447+0000 7f62a9800640 1 -- 172.21.15.143:0/2448881880 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f629c02c5f0 con 0x7f629c1051e0 2024-01-16T15:00:12.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.450+0000 7f2f2f800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f2806ca30 0x7f2f280adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.450+0000 7f6293e00640 1 -- 172.21.15.143:0/2448881880 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f626c005250 con 0x7f629c1051e0 2024-01-16T15:00:12.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.450+0000 7f2f2f800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f28071950 0x7f2f28160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.451+0000 7f2f2f800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f2814b240 0x7f2f281449e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.451+0000 7f2f2f800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f281432e0 con 0x7f2f2814b240 2024-01-16T15:00:12.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.451+0000 7f2f2f800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f280ae520 con 0x7f2f2806ca30 2024-01-16T15:00:12.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.451+0000 7f2f2f800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f28143530 con 0x7f2f28071950 2024-01-16T15:00:12.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.451+0000 7f2f2ce00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f2806ca30 0x7f2f280adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.452+0000 7f2f27e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f28071950 0x7f2f28160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.452+0000 7f2f27e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f28071950 0x7f2f28160ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:57652/0 (socket says 172.21.15.143:57652) 2024-01-16T15:00:12.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.452+0000 7f2f27e00640 1 -- 172.21.15.143:0/3544342078 learned_addr learned my addr 172.21.15.143:0/3544342078 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:12.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.452+0000 7f2f2d800640 1 --2- 172.21.15.143:0/3544342078 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f2814b240 0x7f2f281449e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.453+0000 7f2f27e00640 1 -- 172.21.15.143:0/3544342078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f2806ca30 msgr2=0x7f2f280adfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.453+0000 7f2f27e00640 1 --2- 172.21.15.143:0/3544342078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f2806ca30 0x7f2f280adfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.453+0000 7f2f27e00640 1 -- 172.21.15.143:0/3544342078 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f2814b240 msgr2=0x7f2f281449e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.453+0000 7f2f27e00640 1 --2- 172.21.15.143:0/3544342078 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f2814b240 0x7f2f281449e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.453+0000 7f2f27e00640 1 -- 172.21.15.143:0/3544342078 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f18009400 con 0x7f2f28071950 2024-01-16T15:00:12.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.454+0000 7f2f27e00640 1 --2- 172.21.15.143:0/3544342078 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f28071950 0x7f2f28160ae0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2f18009890 tx=0x7f2f180046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:12.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.452+0000 7f629aa00640 1 -- 172.21.15.143:0/2448881880 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f62a4074940 con 0x7f629c1051e0 2024-01-16T15:00:12.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.455+0000 7f2f24c00640 1 -- 172.21.15.143:0/3544342078 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f18031520 con 0x7f2f28071950 2024-01-16T15:00:12.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.455+0000 7f2f24c00640 1 -- 172.21.15.143:0/3544342078 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f18033b70 con 0x7f2f28071950 2024-01-16T15:00:12.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.455+0000 7f2f24c00640 1 -- 172.21.15.143:0/3544342078 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1803d410 con 0x7f2f28071950 2024-01-16T15:00:12.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.456+0000 7f2f2f800640 1 -- 172.21.15.143:0/3544342078 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f28143810 con 0x7f2f28071950 2024-01-16T15:00:12.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.455+0000 7f629aa00640 1 --2- 172.21.15.143:0/2448881880 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f627c06b2c0 0x7f627c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.455+0000 7f629aa00640 1 -- 172.21.15.143:0/2448881880 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62a4115fb0 con 0x7f629c1051e0 2024-01-16T15:00:12.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.456+0000 7f2f2f800640 1 -- 172.21.15.143:0/3544342078 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f28143dd0 con 0x7f2f28071950 2024-01-16T15:00:12.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.457+0000 7f62a2a00640 1 --2- 172.21.15.143:0/2448881880 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f627c06b2c0 0x7f627c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.458+0000 7f2f2f800640 1 -- 172.21.15.143:0/3544342078 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f2804e840 con 0x7f2f28071950 2024-01-16T15:00:12.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.458+0000 7f62a2a00640 1 --2- 172.21.15.143:0/2448881880 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f627c06b2c0 0x7f627c06d780 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f628c001530 tx=0x7f628c007a40 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:12.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.465+0000 7f629aa00640 1 -- 172.21.15.143:0/2448881880 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f62a40e1140 con 0x7f629c1051e0 2024-01-16T15:00:12.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.464+0000 7f2f24c00640 1 -- 172.21.15.143:0/3544342078 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f2f1803a050 con 0x7f2f28071950 2024-01-16T15:00:12.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.466+0000 7f2f24c00640 1 --2- 172.21.15.143:0/3544342078 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f2f0006b310 0x7f2f0006d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.466+0000 7f2f24c00640 1 -- 172.21.15.143:0/3544342078 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2f1807a7d0 con 0x7f2f28071950 2024-01-16T15:00:12.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.467+0000 7f2f2ce00640 1 --2- 172.21.15.143:0/3544342078 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f2f0006b310 0x7f2f0006d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.468+0000 7f2f2ce00640 1 --2- 172.21.15.143:0/3544342078 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f2f0006b310 0x7f2f0006d7d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2f281618a0 tx=0x7f2f200091b0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:12.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.476+0000 7f2f24c00640 1 -- 172.21.15.143:0/3544342078 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2f180772a0 con 0x7f2f28071950 2024-01-16T15:00:12.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.485+0000 7f72e4600640 1 Processor -- start 2024-01-16T15:00:12.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.486+0000 7f72e4600640 1 -- start start 2024-01-16T15:00:12.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.488+0000 7f72e4600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f72dc15d780 0x7f72dc0b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.488+0000 7f72e4600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f72dc0d3400 0x7f72dc0d3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.488+0000 7f72e4600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f72dc0d1040 0x7f72dc0d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.488+0000 7f72e4600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72dc0ce4f0 con 0x7f72dc0d3400 2024-01-16T15:00:12.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.488+0000 7f72e4600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72dc0d3da0 con 0x7f72dc15d780 2024-01-16T15:00:12.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.488+0000 7f72e4600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc0b36c0 con 0x7f72dc0d1040 2024-01-16T15:00:12.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.488+0000 7f72e1200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f72dc0d3400 0x7f72dc0d3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.488+0000 7f72e1200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f72dc0d3400 0x7f72dc0d3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42800/0 (socket says 172.21.15.143:42800) 2024-01-16T15:00:12.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.488+0000 7f72e1200640 1 -- 172.21.15.143:0/3898789791 learned_addr learned my addr 172.21.15.143:0/3898789791 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:12.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.490+0000 7f72e1c00640 1 --2- 172.21.15.143:0/3898789791 >> v2:172.21.15.204:3300/0 conn(0x7f72dc15d780 0x7f72dc0b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.490+0000 7f72e2600640 1 --2- 172.21.15.143:0/3898789791 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f72dc0d1040 0x7f72dc0d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.491+0000 7f72e1200640 1 -- 172.21.15.143:0/3898789791 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f72dc0d1040 msgr2=0x7f72dc0d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:12.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.491+0000 7f72e1200640 1 --2- 172.21.15.143:0/3898789791 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f72dc0d1040 0x7f72dc0d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.491+0000 7f72e1200640 1 -- 172.21.15.143:0/3898789791 >> v2:172.21.15.204:3300/0 conn(0x7f72dc15d780 msgr2=0x7f72dc0b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:12.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.491+0000 7f72e1200640 1 --2- 172.21.15.143:0/3898789791 >> v2:172.21.15.204:3300/0 conn(0x7f72dc15d780 0x7f72dc0b3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.491+0000 7f72e1200640 1 -- 172.21.15.143:0/3898789791 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72dc0d1d30 con 0x7f72dc0d3400 2024-01-16T15:00:12.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.493+0000 7f72e1200640 1 --2- 172.21.15.143:0/3898789791 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72dc0d3400 0x7f72dc0d3860 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f72dc0b6060 tx=0x7f72c402b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea978c12dc7ff943 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:12.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.494+0000 7f72dbe00640 1 -- 172.21.15.143:0/3898789791 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c4031740 con 0x7f72dc0d3400 2024-01-16T15:00:12.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.495+0000 7f72dbe00640 1 -- 172.21.15.143:0/3898789791 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c40318a0 con 0x7f72dc0d3400 2024-01-16T15:00:12.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.495+0000 7f72dbe00640 1 -- 172.21.15.143:0/3898789791 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c4031c30 con 0x7f72dc0d3400 2024-01-16T15:00:12.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.496+0000 7f72e4600640 1 -- 172.21.15.143:0/3898789791 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72dc0d3400 msgr2=0x7f72dc0d3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.496+0000 7f72e4600640 1 --2- 172.21.15.143:0/3898789791 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72dc0d3400 0x7f72dc0d3860 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f72dc0b6060 tx=0x7f72c402b7e0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.497+0000 7f72e4600640 1 -- 172.21.15.143:0/3898789791 shutdown_connections 2024-01-16T15:00:12.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.497+0000 7f72e4600640 1 --2- 172.21.15.143:0/3898789791 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f72dc0d1040 0x7f72dc0d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.497+0000 7f72e4600640 1 --2- 172.21.15.143:0/3898789791 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72dc0d3400 0x7f72dc0d3860 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.497+0000 7f72e4600640 1 --2- 172.21.15.143:0/3898789791 >> v2:172.21.15.204:3300/0 conn(0x7f72dc15d780 0x7f72dc0b3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.497+0000 7f72e4600640 1 -- 172.21.15.143:0/3898789791 >> 172.21.15.143:0/3898789791 conn(0x7f72dc08cec0 msgr2=0x7f72dc09c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:12.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.497+0000 7f72e4600640 1 -- 172.21.15.143:0/3898789791 shutdown_connections 2024-01-16T15:00:12.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.497+0000 7f72e4600640 1 -- 172.21.15.143:0/3898789791 wait complete. 2024-01-16T15:00:12.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.498+0000 7f5c77200640 1 Processor -- start 2024-01-16T15:00:12.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.498+0000 7f5c77200640 1 -- start start 2024-01-16T15:00:12.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.500+0000 7f5c77200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c7015d590 0x7f5c700cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.500+0000 7f5c77200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5c700d3940 0x7f5c70166f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.500+0000 7f5c77200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c700d1a10 con 0x7f5c700b9000 2024-01-16T15:00:12.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.501+0000 7f5c75e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c7015d590 0x7f5c700cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.501+0000 7f5c75e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c7015d590 0x7f5c700cfa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:57674/0 (socket says 172.21.15.143:57674) 2024-01-16T15:00:12.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.501+0000 7f5c75e00640 1 -- 172.21.15.143:0/3165132360 learned_addr learned my addr 172.21.15.143:0/3165132360 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:12.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.501+0000 7f5c76800640 1 --2- 172.21.15.143:0/3165132360 >> v2:172.21.15.204:3300/0 conn(0x7f5c700d3940 0x7f5c70166f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.500+0000 7f72e4600640 1 Processor -- start 2024-01-16T15:00:12.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.500+0000 7f72e4600640 1 -- start start 2024-01-16T15:00:12.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.501+0000 7f5c77200640 1 -- 172.21.15.143:0/3165132360 --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c700d04c0 con 0x7f5c700d3940 2024-01-16T15:00:12.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.501+0000 7f5c77200640 1 -- 172.21.15.143:0/3165132360 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c700d07a0 con 0x7f5c7015d590 2024-01-16T15:00:12.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.502+0000 7f72e4600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72dc0d1040 0x7f72dc1459c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.502+0000 7f72e4600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f72dc0d3400 0x7f72dc0c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.502+0000 7f72e4600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72dc15d780 0x7f72dc149660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.502+0000 7f72e4600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc145fa0 con 0x7f72dc15d780 2024-01-16T15:00:12.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.502+0000 7f72e4600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc0cf4a0 con 0x7f72dc0d1040 2024-01-16T15:00:12.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.502+0000 7f72e4600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc0ceb70 con 0x7f72dc0d3400 2024-01-16T15:00:12.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.502+0000 7f72e1c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72dc0d1040 0x7f72dc1459c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.502+0000 7f5c74a00640 1 -- 172.21.15.143:0/3165132360 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1502003480 0 0) 0x7f5c700d1a10 con 0x7f5c700b9000 2024-01-16T15:00:12.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.502+0000 7f72e1c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72dc0d1040 0x7f72dc1459c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:60738/0 (socket says 172.21.15.143:60738) 2024-01-16T15:00:12.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.502+0000 7f72e1c00640 1 -- 172.21.15.143:0/1857962963 learned_addr learned my addr 172.21.15.143:0/1857962963 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:12.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.503+0000 7f72e1200640 1 --2- 172.21.15.143:0/1857962963 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f72dc0d3400 0x7f72dc0c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.503+0000 7f72e2600640 1 --2- 172.21.15.143:0/1857962963 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72dc15d780 0x7f72dc149660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.503+0000 7f72e1c00640 1 -- 172.21.15.143:0/1857962963 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f72dc0d3400 msgr2=0x7f72dc0c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.502+0000 7f5c75e00640 1 -- 172.21.15.143:0/3165132360 >> v2:172.21.15.204:3300/0 conn(0x7f5c700d3940 msgr2=0x7f5c70166f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:12.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.502+0000 7f5c75e00640 1 --2- 172.21.15.143:0/3165132360 >> v2:172.21.15.204:3300/0 conn(0x7f5c700d3940 0x7f5c70166f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.502+0000 7f5c75e00640 1 -- 172.21.15.143:0/3165132360 >> v1:172.21.15.204:6789/0 conn(0x7f5c700b9000 legacy=0x7f5c700d3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.502+0000 7f5c75e00640 1 -- 172.21.15.143:0/3165132360 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c700d16a0 con 0x7f5c7015d590 2024-01-16T15:00:12.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.503+0000 7f72e1c00640 1 --2- 172.21.15.143:0/1857962963 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f72dc0d3400 0x7f72dc0c9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.503+0000 7f72e1c00640 1 -- 172.21.15.143:0/1857962963 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72dc15d780 msgr2=0x7f72dc149660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.504+0000 7f72e1c00640 1 --2- 172.21.15.143:0/1857962963 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72dc15d780 0x7f72dc149660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.504+0000 7f72e1c00640 1 -- 172.21.15.143:0/1857962963 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c40094d0 con 0x7f72dc0d1040 2024-01-16T15:00:12.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.504+0000 7f72e2600640 1 --2- 172.21.15.143:0/1857962963 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72dc15d780 0x7f72dc149660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T15:00:12.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.504+0000 7f5c75e00640 1 --2- 172.21.15.143:0/3165132360 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c7015d590 0x7f5c700cfa40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5c58009870 tx=0x7f5c5802b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=112b21d108c60cad server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:12.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.504+0000 7f72e1c00640 1 --2- 172.21.15.143:0/1857962963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72dc0d1040 0x7f72dc1459c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f72d400efc0 tx=0x7f72d400c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:12.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.505+0000 7f5c74a00640 1 -- 172.21.15.143:0/3165132360 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c58031870 con 0x7f5c7015d590 2024-01-16T15:00:12.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.505+0000 7f5c74a00640 1 -- 172.21.15.143:0/3165132360 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c580319d0 con 0x7f5c7015d590 2024-01-16T15:00:12.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.505+0000 7f5c74a00640 1 -- 172.21.15.143:0/3165132360 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c58031da0 con 0x7f5c7015d590 2024-01-16T15:00:12.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.505+0000 7f72d9600640 1 -- 172.21.15.143:0/1857962963 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d4009b80 con 0x7f72dc0d1040 2024-01-16T15:00:12.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.505+0000 7f72d9600640 1 -- 172.21.15.143:0/1857962963 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d400f040 con 0x7f72dc0d1040 2024-01-16T15:00:12.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.505+0000 7f72d9600640 1 -- 172.21.15.143:0/1857962963 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d4009b80 con 0x7f72dc0d1040 2024-01-16T15:00:12.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.505+0000 7f65dbc00640 1 Processor -- start 2024-01-16T15:00:12.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.505+0000 7f72e4600640 1 -- 172.21.15.143:0/1857962963 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72dc0cee50 con 0x7f72dc0d1040 2024-01-16T15:00:12.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.505+0000 7f65dbc00640 1 -- start start 2024-01-16T15:00:12.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.506+0000 7f5c77200640 1 -- 172.21.15.143:0/3165132360 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c7015d590 msgr2=0x7f5c700cfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.506+0000 7f5c77200640 1 --2- 172.21.15.143:0/3165132360 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c7015d590 0x7f5c700cfa40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5c58009870 tx=0x7f5c5802b910 comp rx=0 tx=0).stop 2024-01-16T15:00:12.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.506+0000 7f5c77200640 1 -- 172.21.15.143:0/3165132360 shutdown_connections 2024-01-16T15:00:12.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.506+0000 7f5c77200640 1 --2- 172.21.15.143:0/3165132360 >> v2:172.21.15.204:3300/0 conn(0x7f5c700d3940 0x7f5c70166f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.506+0000 7f5c77200640 1 --2- 172.21.15.143:0/3165132360 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c7015d590 0x7f5c700cfa40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.506+0000 7f5c77200640 1 -- 172.21.15.143:0/3165132360 >> 172.21.15.143:0/3165132360 conn(0x7f5c70061550 msgr2=0x7f5c7006b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:12.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.507+0000 7f5c77200640 1 -- 172.21.15.143:0/3165132360 shutdown_connections 2024-01-16T15:00:12.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.507+0000 7f72e4600640 1 -- 172.21.15.143:0/1857962963 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72dc153ca0 con 0x7f72dc0d1040 2024-01-16T15:00:12.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.507+0000 7fdb18600640 1 Processor -- start 2024-01-16T15:00:12.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.507+0000 7f65dbc00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f65d40cc470 0x7f65d40abfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.508+0000 7f65dbc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65d40ce640 0x7f65d40b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.508+0000 7f65d9200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f65d40cc470 0x7f65d40abfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.508+0000 7f65dbc00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f65d40cf450 0x7f65d40cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.508+0000 7f65dbc00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65d40cfe20 con 0x7f65d40cc470 2024-01-16T15:00:12.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.508+0000 7f65dbc00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65d406c560 con 0x7f65d40cf450 2024-01-16T15:00:12.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.508+0000 7f65dbc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d40ce270 con 0x7f65d40ce640 2024-01-16T15:00:12.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.508+0000 7f72e4600640 1 -- 172.21.15.143:0/1857962963 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72dc067470 con 0x7f72dc0d1040 2024-01-16T15:00:12.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.507+0000 7f5c77200640 1 -- 172.21.15.143:0/3165132360 wait complete. 2024-01-16T15:00:12.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.510+0000 7f5c77200640 1 Processor -- start 2024-01-16T15:00:12.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.509+0000 7fb444600640 1 Processor -- start 2024-01-16T15:00:12.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.507+0000 7fdb18600640 1 -- start start 2024-01-16T15:00:12.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.509+0000 7fdb18600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb100ce640 0x7fdb100abfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.509+0000 7fdb15200640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fdb100cfb60 0x7fdb1006d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54166/0 (socket says 172.21.15.143:54166) 2024-01-16T15:00:12.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.510+0000 7fdb15200640 1 -- 172.21.15.143:0/2448985605 learned_addr learned my addr 172.21.15.143:0/2448985605 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:12.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.510+0000 7fdb18600640 1 -- 172.21.15.143:0/2448985605 --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb100ce270 con 0x7fdb100cfb60 2024-01-16T15:00:12.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.510+0000 7fdb18600640 1 -- 172.21.15.143:0/2448985605 --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb1006d0a0 con 0x7fdb100cc470 2024-01-16T15:00:12.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.510+0000 7fdb18600640 1 -- 172.21.15.143:0/2448985605 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1006c560 con 0x7fdb100ce640 2024-01-16T15:00:12.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.508+0000 7f65d9200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f65d40cc470 0x7f65d40abfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42824/0 (socket says 172.21.15.143:42824) 2024-01-16T15:00:12.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.508+0000 7f65d9200640 1 -- 172.21.15.143:0/833385214 learned_addr learned my addr 172.21.15.143:0/833385214 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:12.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.508+0000 7f65d3e00640 1 --2- 172.21.15.143:0/833385214 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65d40ce640 0x7f65d40b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.508+0000 7f65d9c00640 1 --2- 172.21.15.143:0/833385214 >> v2:172.21.15.204:3300/0 conn(0x7f65d40cf450 0x7f65d40cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.510+0000 7f65d3e00640 1 -- 172.21.15.143:0/833385214 >> v2:172.21.15.204:3300/0 conn(0x7f65d40cf450 msgr2=0x7f65d40cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:12.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.510+0000 7f65d3e00640 1 --2- 172.21.15.143:0/833385214 >> v2:172.21.15.204:3300/0 conn(0x7f65d40cf450 0x7f65d40cf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.510+0000 7f65d3e00640 1 -- 172.21.15.143:0/833385214 >> v2:172.21.15.143:3300/0 conn(0x7f65d40cc470 msgr2=0x7f65d40abfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:12.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.510+0000 7fdb16600640 1 --2- 172.21.15.143:0/2448985605 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb100ce640 0x7fdb100abfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.510+0000 7fdb0fe00640 1 -- 172.21.15.143:0/2448985605 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3015751755 0 0) 0x7fdb100ce270 con 0x7fdb100cfb60 2024-01-16T15:00:12.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.510+0000 7fdb0fe00640 1 -- 172.21.15.143:0/2448985605 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdaf0003520 con 0x7fdb100cfb60 2024-01-16T15:00:12.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.510+0000 7fb444600640 1 -- start start 2024-01-16T15:00:12.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.510+0000 7f5c77200640 1 -- start start 2024-01-16T15:00:12.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.511+0000 7fdb0fe00640 1 -- 172.21.15.143:0/2448985605 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1360706255 0 0) 0x7fdb1006d0a0 con 0x7fdb100cc470 2024-01-16T15:00:12.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.511+0000 7fdb0fe00640 1 -- 172.21.15.143:0/2448985605 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb100ce270 con 0x7fdb100cc470 2024-01-16T15:00:12.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.512+0000 7f5c77200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c700b9000 0x7f5c7006dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.512+0000 7f5c77200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c700d3940 0x7f5c70203c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.512+0000 7fb444600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb43c0ce120 0x7fb43c0cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.512+0000 7fb444600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb43c159e30 con 0x7fb43c0b0ce0 2024-01-16T15:00:12.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.512+0000 7fb444600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb43c0cc690 con 0x7fb43c0b0380 2024-01-16T15:00:12.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.512+0000 7fb444600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c0cc970 con 0x7fb43c0ce120 2024-01-16T15:00:12.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.512+0000 7f5c77200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c7015d590 0x7f5c700b7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.511+0000 7f72d9600640 1 -- 172.21.15.143:0/1857962963 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f72d4014480 con 0x7f72dc0d1040 2024-01-16T15:00:12.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.511+0000 7f65d3e00640 1 --2- 172.21.15.143:0/833385214 >> v2:172.21.15.143:3300/0 conn(0x7f65d40cc470 0x7f65d40abfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.511+0000 7f65d3e00640 1 -- 172.21.15.143:0/833385214 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d40b4200 con 0x7f65d40ce640 2024-01-16T15:00:12.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.512+0000 7f72d9600640 1 --2- 172.21.15.143:0/1857962963 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f72b406b320 0x7f72b406d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.512+0000 7f65d9200640 1 --2- 172.21.15.143:0/833385214 >> v2:172.21.15.143:3300/0 conn(0x7f65d40cc470 0x7f65d40abfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T15:00:12.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.511+0000 7fdb16600640 1 -- 172.21.15.143:0/2448985605 >> v1:172.21.15.204:6789/0 conn(0x7fdb100cc470 legacy=0x7fdb100cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.511+0000 7fdb0fe00640 1 -- 172.21.15.143:0/2448985605 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2165509573 0 0) 0x7fdaf0003520 con 0x7fdb100cfb60 2024-01-16T15:00:12.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.511+0000 7fdb16600640 1 -- 172.21.15.143:0/2448985605 >> v1:172.21.15.143:6789/0 conn(0x7fdb100cfb60 legacy=0x7fdb1006d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.511+0000 7fdb16600640 1 -- 172.21.15.143:0/2448985605 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb1006c770 con 0x7fdb100ce640 2024-01-16T15:00:12.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.513+0000 7f5c77200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c702034d0 con 0x7f5c7015d590 2024-01-16T15:00:12.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.513+0000 7f5c77200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c70203770 con 0x7f5c700b9000 2024-01-16T15:00:12.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.513+0000 7f5c77200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c70070440 con 0x7f5c700d3940 2024-01-16T15:00:12.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.513+0000 7fb442600640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb43c0b0ce0 0x7fb43c0b1170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54174/0 (socket says 172.21.15.143:54174) 2024-01-16T15:00:12.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.513+0000 7fb442600640 1 -- 172.21.15.143:0/198116297 learned_addr learned my addr 172.21.15.143:0/198116297 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:12.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.513+0000 7fb441c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb43c0ce120 0x7fb43c0cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.513+0000 7f5c75e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c700b9000 0x7f5c7006dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.513+0000 7f5c75400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c700d3940 0x7f5c70203c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.513+0000 7f5c76800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c7015d590 0x7f5c700b7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.513+0000 7f5c75e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c700b9000 0x7f5c7006dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:60750/0 (socket says 172.21.15.143:60750) 2024-01-16T15:00:12.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.513+0000 7f5c75e00640 1 -- 172.21.15.143:0/860947532 learned_addr learned my addr 172.21.15.143:0/860947532 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:12.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.512+0000 7f72d9600640 1 -- 172.21.15.143:0/1857962963 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f72d40898c0 con 0x7f72dc0d1040 2024-01-16T15:00:12.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.513+0000 7f65d3e00640 1 --2- 172.21.15.143:0/833385214 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65d40ce640 0x7f65d40b3cc0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f65d40cd610 tx=0x7f65bc02b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ee2aa564ab1de9d server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:12.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.513+0000 7fdb16600640 1 --2- 172.21.15.143:0/2448985605 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb100ce640 0x7fdb100abfc0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdaf80097d0 tx=0x7fdaf802b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b90c7143db9154a9 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:12.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.514+0000 7f5c76800640 1 -- 172.21.15.143:0/860947532 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c700d3940 msgr2=0x7f5c70203c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.514+0000 7f5c76800640 1 --2- 172.21.15.143:0/860947532 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c700d3940 0x7f5c70203c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.514+0000 7f5c76800640 1 -- 172.21.15.143:0/860947532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c700b9000 msgr2=0x7f5c7006dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.514+0000 7f5c76800640 1 --2- 172.21.15.143:0/860947532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c700b9000 0x7f5c7006dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.514+0000 7f5c76800640 1 -- 172.21.15.143:0/860947532 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c580094d0 con 0x7f5c7015d590 2024-01-16T15:00:12.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.514+0000 7fb441c00640 1 -- 172.21.15.143:0/198116297 >> v1:172.21.15.204:6789/0 conn(0x7fb43c0b0380 legacy=0x7fb43c159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.514+0000 7fb441c00640 1 -- 172.21.15.143:0/198116297 >> v1:172.21.15.143:6789/0 conn(0x7fb43c0b0ce0 legacy=0x7fb43c0b1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.514+0000 7f72e1200640 1 --2- 172.21.15.143:0/1857962963 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f72b406b320 0x7f72b406d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.514+0000 7f5c76800640 1 --2- 172.21.15.143:0/860947532 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c7015d590 0x7f5c700b7cb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5c700d3df0 tx=0x7f5c6800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:12.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.514+0000 7fb441c00640 1 -- 172.21.15.143:0/198116297 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb43c0b2180 con 0x7fb43c0ce120 2024-01-16T15:00:12.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.515+0000 7f5c6e000640 1 -- 172.21.15.143:0/860947532 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c68009c00 con 0x7f5c7015d590 2024-01-16T15:00:12.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.515+0000 7f5c6e000640 1 -- 172.21.15.143:0/860947532 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6800f040 con 0x7f5c7015d590 2024-01-16T15:00:12.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.515+0000 7f5c6e000640 1 -- 172.21.15.143:0/860947532 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c680146c0 con 0x7f5c7015d590 2024-01-16T15:00:12.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.515+0000 7f5c77200640 1 -- 172.21.15.143:0/860947532 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c7006e3b0 con 0x7f5c7015d590 2024-01-16T15:00:12.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.516+0000 7fdb0fe00640 1 -- 172.21.15.143:0/2448985605 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf802f050 con 0x7fdb100ce640 2024-01-16T15:00:12.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.515+0000 7f5c77200640 1 -- 172.21.15.143:0/860947532 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c7006e970 con 0x7f5c7015d590 2024-01-16T15:00:12.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.516+0000 7fdb0fe00640 1 -- 172.21.15.143:0/2448985605 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaf8031500 con 0x7fdb100ce640 2024-01-16T15:00:12.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.516+0000 7fdb0fe00640 1 -- 172.21.15.143:0/2448985605 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb00001670 con 0x7fdb100ce640 2024-01-16T15:00:12.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.516+0000 7f65d3400640 1 -- 172.21.15.143:0/833385214 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65bc0317c0 con 0x7f65d40ce640 2024-01-16T15:00:12.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.516+0000 7f65d3400640 1 -- 172.21.15.143:0/833385214 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65bc031920 con 0x7f65d40ce640 2024-01-16T15:00:12.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.516+0000 7f65d3400640 1 -- 172.21.15.143:0/833385214 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65bc031cb0 con 0x7f65d40ce640 2024-01-16T15:00:12.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.517+0000 7f72e1200640 1 --2- 172.21.15.143:0/1857962963 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f72b406b320 0x7f72b406d7e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f72c4035000 tx=0x7f72c4008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:12.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.517+0000 7fdb18600640 1 -- 172.21.15.143:0/2448985605 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb100ce640 msgr2=0x7fdb100abfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.517+0000 7fdb18600640 1 --2- 172.21.15.143:0/2448985605 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb100ce640 0x7fdb100abfc0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdaf80097d0 tx=0x7fdaf802b820 comp rx=0 tx=0).stop 2024-01-16T15:00:12.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.517+0000 7f5c77200640 1 -- 172.21.15.143:0/860947532 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c7006eb90 con 0x7f5c7015d590 2024-01-16T15:00:12.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.517+0000 7fdb18600640 1 -- 172.21.15.143:0/2448985605 shutdown_connections 2024-01-16T15:00:12.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.517+0000 7fdb18600640 1 --2- 172.21.15.143:0/2448985605 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb100ce640 0x7fdb100abfc0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.517+0000 7fdb18600640 1 -- 172.21.15.143:0/2448985605 >> 172.21.15.143:0/2448985605 conn(0x7fdb1006a920 msgr2=0x7fdb1006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:12.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.517+0000 7fb441c00640 1 --2- 172.21.15.143:0/198116297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb43c0ce120 0x7fb43c0cbd20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb42c0097d0 tx=0x7fb42c02b9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e7fc87f548aecf2 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:12.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.518+0000 7fdb18600640 1 -- 172.21.15.143:0/2448985605 shutdown_connections 2024-01-16T15:00:12.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.518+0000 7fdb18600640 1 -- 172.21.15.143:0/2448985605 wait complete. 2024-01-16T15:00:12.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.518+0000 7f65dbc00640 1 -- 172.21.15.143:0/833385214 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65d40ce640 msgr2=0x7f65d40b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.518+0000 7f65dbc00640 1 --2- 172.21.15.143:0/833385214 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65d40ce640 0x7f65d40b3cc0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f65d40cd610 tx=0x7f65bc02b860 comp rx=0 tx=0).stop 2024-01-16T15:00:12.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.518+0000 7fb43be00640 1 -- 172.21.15.143:0/198116297 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c030890 con 0x7fb43c0ce120 2024-01-16T15:00:12.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.518+0000 7fb43be00640 1 -- 172.21.15.143:0/198116297 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb42c0309f0 con 0x7fb43c0ce120 2024-01-16T15:00:12.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.518+0000 7fb43be00640 1 -- 172.21.15.143:0/198116297 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c030d80 con 0x7fb43c0ce120 2024-01-16T15:00:12.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.519+0000 7fb444600640 1 -- 172.21.15.143:0/198116297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb43c0ce120 msgr2=0x7fb43c0cbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.519+0000 7fb444600640 1 --2- 172.21.15.143:0/198116297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb43c0ce120 0x7fb43c0cbd20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb42c0097d0 tx=0x7fb42c02b9d0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.519+0000 7f65dbc00640 1 -- 172.21.15.143:0/833385214 shutdown_connections 2024-01-16T15:00:12.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.519+0000 7f65dbc00640 1 --2- 172.21.15.143:0/833385214 >> v2:172.21.15.204:3300/0 conn(0x7f65d40cf450 0x7f65d40cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.519+0000 7f65dbc00640 1 --2- 172.21.15.143:0/833385214 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65d40ce640 0x7f65d40b3cc0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.519+0000 7f65dbc00640 1 --2- 172.21.15.143:0/833385214 >> v2:172.21.15.143:3300/0 conn(0x7f65d40cc470 0x7f65d40abfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.519+0000 7f65dbc00640 1 -- 172.21.15.143:0/833385214 >> 172.21.15.143:0/833385214 conn(0x7f65d406a920 msgr2=0x7f65d406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:12.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.519+0000 7fb444600640 1 -- 172.21.15.143:0/198116297 shutdown_connections 2024-01-16T15:00:12.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.520+0000 7f65dbc00640 1 -- 172.21.15.143:0/833385214 shutdown_connections 2024-01-16T15:00:12.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.519+0000 7fb444600640 1 --2- 172.21.15.143:0/198116297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb43c0ce120 0x7fb43c0cbd20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.520+0000 7fb444600640 1 -- 172.21.15.143:0/198116297 >> 172.21.15.143:0/198116297 conn(0x7fb43c0b4500 msgr2=0x7fb43c0b4910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:12.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.520+0000 7f65dbc00640 1 -- 172.21.15.143:0/833385214 wait complete. 2024-01-16T15:00:12.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.519+0000 7f5c6e000640 1 -- 172.21.15.143:0/860947532 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f5c68014820 con 0x7f5c7015d590 2024-01-16T15:00:12.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.520+0000 7fdb18600640 1 Processor -- start 2024-01-16T15:00:12.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.520+0000 7f5c6e000640 1 --2- 172.21.15.143:0/860947532 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f5c4806b390 0x7f5c4806d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.520+0000 7fdb18600640 1 -- start start 2024-01-16T15:00:12.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.521+0000 7f5c6e000640 1 -- 172.21.15.143:0/860947532 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c6808a860 con 0x7f5c7015d590 2024-01-16T15:00:12.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.521+0000 7fb444600640 1 -- 172.21.15.143:0/198116297 shutdown_connections 2024-01-16T15:00:12.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.521+0000 7fb444600640 1 -- 172.21.15.143:0/198116297 wait complete. 2024-01-16T15:00:12.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.522+0000 7f65dbc00640 1 Processor -- start 2024-01-16T15:00:12.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.522+0000 7f5c75e00640 1 --2- 172.21.15.143:0/860947532 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f5c4806b390 0x7f5c4806d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.522+0000 7f65dbc00640 1 -- start start 2024-01-16T15:00:12.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.522+0000 7fdb18600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb100cc470 0x7fdb10160750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.522+0000 7fdb18600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb100ce640 0x7fdb10164ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.522+0000 7fdb18600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb100cfb60 0x7fdb1015e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.522+0000 7fdb18600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb10160f10 con 0x7fdb100cc470 2024-01-16T15:00:12.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.522+0000 7fdb18600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb10160c90 con 0x7fdb100ce640 2024-01-16T15:00:12.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.522+0000 7fdb18600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb10165400 con 0x7fdb100cfb60 2024-01-16T15:00:12.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.522+0000 7fdb15c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb100cc470 0x7fdb10160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.523+0000 7fdb15c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb100cc470 0x7fdb10160750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42844/0 (socket says 172.21.15.143:42844) 2024-01-16T15:00:12.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.523+0000 7fdb15c00640 1 -- 172.21.15.143:0/1204674242 learned_addr learned my addr 172.21.15.143:0/1204674242 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:12.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.523+0000 7f91e7000640 1 Processor -- start 2024-01-16T15:00:12.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.523+0000 7fdb15200640 1 --2- 172.21.15.143:0/1204674242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb100ce640 0x7fdb10164ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.523+0000 7fdb16600640 1 --2- 172.21.15.143:0/1204674242 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb100cfb60 0x7fdb1015e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.523+0000 7f65dbc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65d40cc470 0x7f65d4156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.524+0000 7f65dbc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65d40ce640 0x7f65d41eb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.524+0000 7f65dbc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65d40cf450 0x7f65d4158f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.524+0000 7f65dbc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d41ebe60 con 0x7f65d40ce640 2024-01-16T15:00:12.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.524+0000 7f65dbc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d41ec140 con 0x7f65d40cf450 2024-01-16T15:00:12.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.524+0000 7f65dbc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d41ec420 con 0x7f65d40cc470 2024-01-16T15:00:12.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.524+0000 7fdb15c00640 1 -- 172.21.15.143:0/1204674242 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb100cfb60 msgr2=0x7fdb1015e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.524+0000 7fdb15c00640 1 --2- 172.21.15.143:0/1204674242 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb100cfb60 0x7fdb1015e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.524+0000 7fdb15c00640 1 -- 172.21.15.143:0/1204674242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb100ce640 msgr2=0x7fdb10164ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.524+0000 7fdb15c00640 1 --2- 172.21.15.143:0/1204674242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb100ce640 0x7fdb10164ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.524+0000 7fdb15c00640 1 -- 172.21.15.143:0/1204674242 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf8009400 con 0x7fdb100cc470 2024-01-16T15:00:12.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.524+0000 7f65d9200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65d40cc470 0x7f65d4156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.524+0000 7f65d3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65d40ce640 0x7f65d41eb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.524+0000 7f65d9c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65d40cf450 0x7f65d4158f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.524+0000 7f65d9c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65d40cf450 0x7f65d4158f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:60776/0 (socket says 172.21.15.143:60776) 2024-01-16T15:00:12.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.524+0000 7f65d9c00640 1 -- 172.21.15.143:0/3395157685 learned_addr learned my addr 172.21.15.143:0/3395157685 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:12.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.525+0000 7fdb15c00640 1 --2- 172.21.15.143:0/1204674242 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb100cc470 0x7fdb10160750 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdb0800c7f0 tx=0x7fdb0800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:12.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.525+0000 7fb444600640 1 Processor -- start 2024-01-16T15:00:12.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.525+0000 7fb444600640 1 -- start start 2024-01-16T15:00:12.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.525+0000 7f65d9c00640 1 -- 172.21.15.143:0/3395157685 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65d40cc470 msgr2=0x7f65d4156b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.525+0000 7f65d9c00640 1 --2- 172.21.15.143:0/3395157685 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65d40cc470 0x7f65d4156b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.525+0000 7f65d9c00640 1 -- 172.21.15.143:0/3395157685 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65d40ce640 msgr2=0x7f65d41eb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.525+0000 7f65d9c00640 1 --2- 172.21.15.143:0/3395157685 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65d40ce640 0x7f65d41eb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.525+0000 7f65d9c00640 1 -- 172.21.15.143:0/3395157685 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c40094d0 con 0x7f65d40cf450 2024-01-16T15:00:12.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.523+0000 7f91e7000640 1 -- start start 2024-01-16T15:00:12.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.525+0000 7f91e7000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91e00b1840 0x7f91e00cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.525+0000 7f91e7000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91e00cc3b0 con 0x7f91e00ce120 2024-01-16T15:00:12.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.525+0000 7f91e7000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91e00cc690 con 0x7f91e0159990 2024-01-16T15:00:12.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.525+0000 7f91e7000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e00cc970 con 0x7f91e00b1840 2024-01-16T15:00:12.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.526+0000 7f5c75e00640 1 --2- 172.21.15.143:0/860947532 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f5c4806b390 0x7f5c4806d850 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5c700b88d0 tx=0x7f5c5803c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:12.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.526+0000 7f91dfe00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f91e0159990 0x7f91e00b0380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:33218/0 (socket says 172.21.15.143:33218) 2024-01-16T15:00:12.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.526+0000 7f91dfe00640 1 -- 172.21.15.143:0/334047929 learned_addr learned my addr 172.21.15.143:0/334047929 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:12.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.526+0000 7f91e5000640 1 --2- 172.21.15.143:0/334047929 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91e00b1840 0x7f91e00cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.527+0000 7f65d9c00640 1 --2- 172.21.15.143:0/3395157685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65d40cf450 0x7f65d4158f10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f65cc00ec10 tx=0x7f65cc00c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:12.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.527+0000 7fb444600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb43c0b0380 0x7fb43c145970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.527+0000 7fb444600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb43c0b0ce0 0x7fb43c159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.527+0000 7fb444600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb43c0ce120 0x7fb43c167590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.527+0000 7f72d9600640 1 -- 172.21.15.143:0/1857962963 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f72d4054750 con 0x7f72dc0d1040 2024-01-16T15:00:12.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.527+0000 7f91dea00640 1 -- 172.21.15.143:0/334047929 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3381677527 0 0) 0x7f91e00cc690 con 0x7f91e0159990 2024-01-16T15:00:12.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.527+0000 7f91dea00640 1 -- 172.21.15.143:0/334047929 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91c0003520 con 0x7f91e0159990 2024-01-16T15:00:12.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.527+0000 7fb444600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c144560 con 0x7fb43c0ce120 2024-01-16T15:00:12.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.527+0000 7fb444600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c168480 con 0x7fb43c0b0380 2024-01-16T15:00:12.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.527+0000 7fb444600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c167be0 con 0x7fb43c0b0ce0 2024-01-16T15:00:12.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.527+0000 7fb441c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb43c0b0380 0x7fb43c145970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.527+0000 7fb441c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb43c0b0380 0x7fb43c145970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:60784/0 (socket says 172.21.15.143:60784) 2024-01-16T15:00:12.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.527+0000 7fb441c00640 1 -- 172.21.15.143:0/1918003465 learned_addr learned my addr 172.21.15.143:0/1918003465 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:12.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.528+0000 7fb441200640 1 --2- 172.21.15.143:0/1918003465 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb43c0b0ce0 0x7fb43c159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.527+0000 7f65d0c00640 1 -- 172.21.15.143:0/3395157685 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65cc00cd20 con 0x7f65d40cf450 2024-01-16T15:00:12.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.527+0000 7f65d0c00640 1 -- 172.21.15.143:0/3395157685 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65cc00f040 con 0x7f65d40cf450 2024-01-16T15:00:12.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.527+0000 7f65d0c00640 1 -- 172.21.15.143:0/3395157685 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65cc0146c0 con 0x7f65d40cf450 2024-01-16T15:00:12.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.527+0000 7f91e5000640 1 -- 172.21.15.143:0/334047929 >> v1:172.21.15.204:6789/0 conn(0x7f91e0159990 legacy=0x7f91e00b0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.527+0000 7f91e5000640 1 -- 172.21.15.143:0/334047929 >> v1:172.21.15.143:6789/0 conn(0x7f91e00ce120 legacy=0x7f91e00b1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.528+0000 7f5c6e000640 1 -- 172.21.15.143:0/860947532 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5c68055760 con 0x7f5c7015d590 2024-01-16T15:00:12.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.528+0000 7f65dbc00640 1 -- 172.21.15.143:0/3395157685 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65bc009400 con 0x7f65d40cf450 2024-01-16T15:00:12.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.528+0000 7f65dbc00640 1 -- 172.21.15.143:0/3395157685 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d41ec9e0 con 0x7f65d40cf450 2024-01-16T15:00:12.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.528+0000 7fb441c00640 1 -- 172.21.15.143:0/1918003465 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb43c0b0ce0 msgr2=0x7fb43c159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.528+0000 7fb441c00640 1 --2- 172.21.15.143:0/1918003465 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb43c0b0ce0 0x7fb43c159ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.528+0000 7fb441c00640 1 -- 172.21.15.143:0/1918003465 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb43c0ce120 msgr2=0x7fb43c167590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.528+0000 7fb441c00640 1 --2- 172.21.15.143:0/1918003465 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb43c0ce120 0x7fb43c167590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.528+0000 7fb441c00640 1 -- 172.21.15.143:0/1918003465 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb42c009400 con 0x7fb43c0b0380 2024-01-16T15:00:12.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.528+0000 7f91e5000640 1 -- 172.21.15.143:0/334047929 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e00b2180 con 0x7f91e00b1840 2024-01-16T15:00:12.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.529+0000 7fdb0d600640 1 -- 172.21.15.143:0/1204674242 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb08003b70 con 0x7fdb100cc470 2024-01-16T15:00:12.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.529+0000 7fdb0d600640 1 -- 172.21.15.143:0/1204674242 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb08003cd0 con 0x7fdb100cc470 2024-01-16T15:00:12.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.529+0000 7fdb0d600640 1 -- 172.21.15.143:0/1204674242 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0800aa70 con 0x7fdb100cc470 2024-01-16T15:00:12.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.529+0000 7fdb18600640 1 -- 172.21.15.143:0/1204674242 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb101649e0 con 0x7fdb100cc470 2024-01-16T15:00:12.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.529+0000 7f91e5000640 1 --2- 172.21.15.143:0/334047929 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91e00b1840 0x7f91e00cbd20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009b40 tx=0x7f91d802d190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1008b0fd154d9e9 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:12.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.529+0000 7fb441c00640 1 --2- 172.21.15.143:0/1918003465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb43c0b0380 0x7fb43c145970 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb42c0098f0 tx=0x7fb42c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:12.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.530+0000 7fb439600640 1 -- 172.21.15.143:0/1918003465 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c03e040 con 0x7fb43c0b0380 2024-01-16T15:00:12.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.530+0000 7fb439600640 1 -- 172.21.15.143:0/1918003465 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb42c0443f0 con 0x7fb43c0b0380 2024-01-16T15:00:12.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.530+0000 7fb439600640 1 -- 172.21.15.143:0/1918003465 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c0447a0 con 0x7fb43c0b0380 2024-01-16T15:00:12.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.530+0000 7f65dbc00640 1 -- 172.21.15.143:0/3395157685 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65d41ecc00 con 0x7f65d40cf450 2024-01-16T15:00:12.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.530+0000 7fdb18600640 1 -- 172.21.15.143:0/1204674242 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb100995c0 con 0x7fdb100cc470 2024-01-16T15:00:12.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.531+0000 7f91dea00640 1 -- 172.21.15.143:0/334047929 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d8035440 con 0x7f91e00b1840 2024-01-16T15:00:12.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.531+0000 7f91dea00640 1 -- 172.21.15.143:0/334047929 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d80355a0 con 0x7f91e00b1840 2024-01-16T15:00:12.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.531+0000 7f91dea00640 1 -- 172.21.15.143:0/334047929 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d8035930 con 0x7f91e00b1840 2024-01-16T15:00:12.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.532+0000 7f91e7000640 1 -- 172.21.15.143:0/334047929 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91e00b1840 msgr2=0x7f91e00cbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.532+0000 7f91e7000640 1 --2- 172.21.15.143:0/334047929 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91e00b1840 0x7f91e00cbd20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009b40 tx=0x7f91d802d190 comp rx=0 tx=0).stop 2024-01-16T15:00:12.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.532+0000 7f91e7000640 1 -- 172.21.15.143:0/334047929 shutdown_connections 2024-01-16T15:00:12.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.532+0000 7f91e7000640 1 --2- 172.21.15.143:0/334047929 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91e00b1840 0x7f91e00cbd20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.532+0000 7f91e7000640 1 -- 172.21.15.143:0/334047929 >> 172.21.15.143:0/334047929 conn(0x7f91e00b4500 msgr2=0x7f91e00b4910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:12.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.532+0000 7f91e7000640 1 -- 172.21.15.143:0/334047929 shutdown_connections 2024-01-16T15:00:12.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.532+0000 7f91e7000640 1 -- 172.21.15.143:0/334047929 wait complete. 2024-01-16T15:00:12.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.532+0000 7fdb18600640 1 -- 172.21.15.143:0/1204674242 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb10067470 con 0x7fdb100cc470 2024-01-16T15:00:12.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.532+0000 7fdb0d600640 1 -- 172.21.15.143:0/1204674242 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fdb0800abd0 con 0x7fdb100cc470 2024-01-16T15:00:12.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.533+0000 7fb444600640 1 -- 172.21.15.143:0/1918003465 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb43c167ec0 con 0x7fb43c0b0380 2024-01-16T15:00:12.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.533+0000 7fb444600640 1 -- 172.21.15.143:0/1918003465 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb43c14d920 con 0x7fb43c0b0380 2024-01-16T15:00:12.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.533+0000 7f65d0c00640 1 -- 172.21.15.143:0/3395157685 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f65cc014820 con 0x7f65d40cf450 2024-01-16T15:00:12.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.534+0000 7fdb0d600640 1 --2- 172.21.15.143:0/1204674242 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fdae806b310 0x7fdae806d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.535+0000 7f91e7000640 1 Processor -- start 2024-01-16T15:00:12.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.534+0000 7f65d0c00640 1 --2- 172.21.15.143:0/3395157685 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f65ac06b390 0x7f65ac06d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.534+0000 7f65d0c00640 1 -- 172.21.15.143:0/3395157685 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f65cc08afb0 con 0x7f65d40cf450 2024-01-16T15:00:12.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.535+0000 7f91e7000640 1 -- start start 2024-01-16T15:00:12.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.535+0000 7fdb15200640 1 --2- 172.21.15.143:0/1204674242 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fdae806b310 0x7fdae806d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.535+0000 7fb444600640 1 -- 172.21.15.143:0/1918003465 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb43c067470 con 0x7fb43c0b0380 2024-01-16T15:00:12.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.536+0000 7f91e7000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91e00b1840 0x7f91e0153930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.536+0000 7f91e7000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91e00ce120 0x7f91e015c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.536+0000 7f91e7000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91e0159990 0x7f91e00cec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.536+0000 7f91e7000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e00acd70 con 0x7f91e0159990 2024-01-16T15:00:12.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.536+0000 7f91e7000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e00ac160 con 0x7f91e00b1840 2024-01-16T15:00:12.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.536+0000 7f91e7000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e00ac440 con 0x7f91e00ce120 2024-01-16T15:00:12.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.537+0000 7fdb15200640 1 --2- 172.21.15.143:0/1204674242 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fdae806b310 0x7fdae806d7d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdb10161070 tx=0x7fdb00009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:12.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.536+0000 7f91e5000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91e0159990 0x7f91e00cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.536+0000 7f91e5000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91e0159990 0x7f91e00cec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42872/0 (socket says 172.21.15.143:42872) 2024-01-16T15:00:12.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.536+0000 7f91e5000640 1 -- 172.21.15.143:0/2668467430 learned_addr learned my addr 172.21.15.143:0/2668467430 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:12.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.537+0000 7f91df400640 1 --2- 172.21.15.143:0/2668467430 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91e00ce120 0x7f91e015c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.537+0000 7fdb0d600640 1 -- 172.21.15.143:0/1204674242 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb08003e40 con 0x7fdb100cc470 2024-01-16T15:00:12.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.537+0000 7f91e5000640 1 -- 172.21.15.143:0/2668467430 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91e00ce120 msgr2=0x7f91e015c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.537+0000 7f91e5000640 1 --2- 172.21.15.143:0/2668467430 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91e00ce120 0x7f91e015c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.537+0000 7f91e5000640 1 -- 172.21.15.143:0/2668467430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91e00b1840 msgr2=0x7f91e0153930 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-16T15:00:12.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.537+0000 7f91e5000640 1 --2- 172.21.15.143:0/2668467430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91e00b1840 0x7f91e0153930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.537+0000 7f91e5000640 1 -- 172.21.15.143:0/2668467430 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d8009770 con 0x7f91e0159990 2024-01-16T15:00:12.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.537+0000 7f91e5000640 1 --2- 172.21.15.143:0/2668467430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91e0159990 0x7f91e00cec90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009a90 tx=0x7f91d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:12.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.538+0000 7f65d9200640 1 --2- 172.21.15.143:0/3395157685 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f65ac06b390 0x7f65ac06d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.539+0000 7f65d9200640 1 --2- 172.21.15.143:0/3395157685 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f65ac06b390 0x7f65ac06d850 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f65d40ad160 tx=0x7f65c4009240 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:12.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.544+0000 7f65d0c00640 1 -- 172.21.15.143:0/3395157685 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f65cc055e00 con 0x7f65d40cf450 2024-01-16T15:00:12.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.545+0000 7f91d7e00640 1 -- 172.21.15.143:0/2668467430 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d8036ea0 con 0x7f91e0159990 2024-01-16T15:00:12.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.545+0000 7f91d7e00640 1 -- 172.21.15.143:0/2668467430 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d8033870 con 0x7f91e0159990 2024-01-16T15:00:12.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.545+0000 7f91d7e00640 1 -- 172.21.15.143:0/2668467430 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d8033c00 con 0x7f91e0159990 2024-01-16T15:00:12.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.545+0000 7f91e7000640 1 -- 172.21.15.143:0/2668467430 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91e00ac720 con 0x7f91e0159990 2024-01-16T15:00:12.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.545+0000 7f91e7000640 1 -- 172.21.15.143:0/2668467430 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91e01543b0 con 0x7f91e0159990 2024-01-16T15:00:12.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.545+0000 7fdb0d600640 1 -- 172.21.15.143:0/1204674242 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdb080547d0 con 0x7fdb100cc470 2024-01-16T15:00:12.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.547+0000 7f91e7000640 1 -- 172.21.15.143:0/2668467430 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91e0067470 con 0x7f91e0159990 2024-01-16T15:00:12.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.549+0000 7f91d7e00640 1 -- 172.21.15.143:0/2668467430 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f91d803c040 con 0x7f91e0159990 2024-01-16T15:00:12.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.548+0000 7fb439600640 1 -- 172.21.15.143:0/1918003465 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fb42c044900 con 0x7fb43c0b0380 2024-01-16T15:00:12.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.549+0000 7fb439600640 1 --2- 172.21.15.143:0/1918003465 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb41406b2f0 0x7fb41406d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.550+0000 7fb439600640 1 -- 172.21.15.143:0/1918003465 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb42c0ac9f0 con 0x7fb43c0b0380 2024-01-16T15:00:12.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.551+0000 7f91d7e00640 1 --2- 172.21.15.143:0/2668467430 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f91b806b310 0x7f91b806d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:12.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.551+0000 7f91d7e00640 1 -- 172.21.15.143:0/2668467430 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f91d80ac700 con 0x7f91e0159990 2024-01-16T15:00:12.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.552+0000 7fb441200640 1 --2- 172.21.15.143:0/1918003465 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb41406b2f0 0x7fb41406d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.552+0000 7f91dfe00640 1 --2- 172.21.15.143:0/2668467430 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f91b806b310 0x7f91b806d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:12.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.554+0000 7fb441200640 1 --2- 172.21.15.143:0/1918003465 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb41406b2f0 0x7fb41406d7b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb43c0cbb90 tx=0x7fb424009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:12.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.554+0000 7f91dfe00640 1 --2- 172.21.15.143:0/2668467430 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f91b806b310 0x7f91b806d7d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f91d0001590 tx=0x7f91d00078f0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:12.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.560+0000 7f91d7e00640 1 -- 172.21.15.143:0/2668467430 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f91d8077800 con 0x7f91e0159990 2024-01-16T15:00:12.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.561+0000 7fb439600640 1 -- 172.21.15.143:0/1918003465 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb42c077980 con 0x7fb43c0b0380 2024-01-16T15:00:12.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.873+0000 7f6293e00640 1 -- 172.21.15.143:0/2448881880 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f626c0057d0 con 0x7f629c1051e0 2024-01-16T15:00:12.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.873+0000 7f2f2f800640 1 -- 172.21.15.143:0/3544342078 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2f280b0500 con 0x7f2f28071950 2024-01-16T15:00:12.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.874+0000 7f629aa00640 1 -- 172.21.15.143:0/2448881880 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f62a40b5020 con 0x7f629c1051e0 2024-01-16T15:00:12.856 INFO:teuthology.orchestra.run.smithi143.stdout:55834574854 2024-01-16T15:00:12.856 INFO:teuthology.orchestra.run.smithi143.stdout:55834574855 2024-01-16T15:00:12.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.874+0000 7f2f24c00640 1 -- 172.21.15.143:0/3544342078 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2f1804d090 con 0x7f2f28071950 2024-01-16T15:00:12.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.878+0000 7f6293e00640 1 -- 172.21.15.143:0/2448881880 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f627c06b2c0 msgr2=0x7f627c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.878+0000 7f2f2f800640 1 -- 172.21.15.143:0/3544342078 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f2f0006b310 msgr2=0x7f2f0006d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.878+0000 7f2f2f800640 1 --2- 172.21.15.143:0/3544342078 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f2f0006b310 0x7f2f0006d7d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2f281618a0 tx=0x7f2f200091b0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.878+0000 7f6293e00640 1 --2- 172.21.15.143:0/2448881880 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f627c06b2c0 0x7f627c06d780 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f628c001530 tx=0x7f628c007a40 comp rx=0 tx=0).stop 2024-01-16T15:00:12.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.878+0000 7f2f2f800640 1 -- 172.21.15.143:0/3544342078 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f28071950 msgr2=0x7f2f28160ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.878+0000 7f6293e00640 1 -- 172.21.15.143:0/2448881880 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f629c1051e0 msgr2=0x7f629c02d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.878+0000 7f2f2f800640 1 --2- 172.21.15.143:0/3544342078 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f28071950 0x7f2f28160ae0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2f18009890 tx=0x7f2f180046c0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.878+0000 7f6293e00640 1 --2- 172.21.15.143:0/2448881880 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f629c1051e0 0x7f629c02d380 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f62a402cee0 tx=0x7f62a406a3b0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.879+0000 7f2f2f800640 1 -- 172.21.15.143:0/3544342078 shutdown_connections 2024-01-16T15:00:12.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.879+0000 7f2f2f800640 1 --2- 172.21.15.143:0/3544342078 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f2f0006b310 0x7f2f0006d7d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.879+0000 7f2f2f800640 1 --2- 172.21.15.143:0/3544342078 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f2814b240 0x7f2f281449e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.879+0000 7f2f2f800640 1 --2- 172.21.15.143:0/3544342078 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f28071950 0x7f2f28160ae0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.879+0000 7f6293e00640 1 -- 172.21.15.143:0/2448881880 shutdown_connections 2024-01-16T15:00:12.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.879+0000 7f2f2f800640 1 --2- 172.21.15.143:0/3544342078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f2806ca30 0x7f2f280adfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.879+0000 7f2f2f800640 1 -- 172.21.15.143:0/3544342078 >> 172.21.15.143:0/3544342078 conn(0x7f2f2806dc30 msgr2=0x7f2f280ae9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:12.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.879+0000 7f6293e00640 1 --2- 172.21.15.143:0/2448881880 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f627c06b2c0 0x7f627c06d780 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.879+0000 7f6293e00640 1 --2- 172.21.15.143:0/2448881880 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f629c1051e0 0x7f629c02d380 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.879+0000 7f6293e00640 1 --2- 172.21.15.143:0/2448881880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f629c04f290 0x7f629c02b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.879+0000 7f6293e00640 1 --2- 172.21.15.143:0/2448881880 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f629c0075b0 0x7f629c1a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.879+0000 7f6293e00640 1 -- 172.21.15.143:0/2448881880 >> 172.21.15.143:0/2448881880 conn(0x7f629c009fb0 msgr2=0x7f629c053f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:12.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.880+0000 7f6293e00640 1 -- 172.21.15.143:0/2448881880 shutdown_connections 2024-01-16T15:00:12.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.880+0000 7f6293e00640 1 -- 172.21.15.143:0/2448881880 wait complete. 2024-01-16T15:00:12.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.880+0000 7f2f2f800640 1 -- 172.21.15.143:0/3544342078 shutdown_connections 2024-01-16T15:00:12.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.880+0000 7f2f2f800640 1 -- 172.21.15.143:0/3544342078 wait complete. 2024-01-16T15:00:12.888 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2024-01-16T15:00:12.889 DEBUG:teuthology.parallel:result is None 2024-01-16T15:00:12.889 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2024-01-16T15:00:12.889 DEBUG:teuthology.parallel:result is None 2024-01-16T15:00:12.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.924+0000 7f65dbc00640 1 -- 172.21.15.143:0/3395157685 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f65d41ec280 con 0x7f65d40cf450 2024-01-16T15:00:12.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.925+0000 7f65d0c00640 1 -- 172.21.15.143:0/3395157685 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f65d41ec280 con 0x7f65d40cf450 2024-01-16T15:00:12.906 INFO:teuthology.orchestra.run.smithi143.stdout:55834574856 2024-01-16T15:00:12.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.928+0000 7f65dbc00640 1 -- 172.21.15.143:0/3395157685 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f65ac06b390 msgr2=0x7f65ac06d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.928+0000 7f65dbc00640 1 --2- 172.21.15.143:0/3395157685 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f65ac06b390 0x7f65ac06d850 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f65d40ad160 tx=0x7f65c4009240 comp rx=0 tx=0).stop 2024-01-16T15:00:12.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.928+0000 7f65dbc00640 1 -- 172.21.15.143:0/3395157685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65d40cf450 msgr2=0x7f65d4158f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.928+0000 7f65dbc00640 1 --2- 172.21.15.143:0/3395157685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65d40cf450 0x7f65d4158f10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f65cc00ec10 tx=0x7f65cc00c490 comp rx=0 tx=0).stop 2024-01-16T15:00:12.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.928+0000 7f65dbc00640 1 -- 172.21.15.143:0/3395157685 shutdown_connections 2024-01-16T15:00:12.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.928+0000 7f65dbc00640 1 --2- 172.21.15.143:0/3395157685 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f65ac06b390 0x7f65ac06d850 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.928+0000 7f65dbc00640 1 --2- 172.21.15.143:0/3395157685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65d40cf450 0x7f65d4158f10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.928+0000 7f65dbc00640 1 --2- 172.21.15.143:0/3395157685 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65d40ce640 0x7f65d41eb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.928+0000 7f65dbc00640 1 --2- 172.21.15.143:0/3395157685 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65d40cc470 0x7f65d4156b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.928+0000 7f65dbc00640 1 -- 172.21.15.143:0/3395157685 >> 172.21.15.143:0/3395157685 conn(0x7f65d406a920 msgr2=0x7f65d40ad2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:12.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.928+0000 7f65dbc00640 1 -- 172.21.15.143:0/3395157685 shutdown_connections 2024-01-16T15:00:12.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.928+0000 7f65dbc00640 1 -- 172.21.15.143:0/3395157685 wait complete. 2024-01-16T15:00:12.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.942+0000 7f91e7000640 1 -- 172.21.15.143:0/2668467430 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f91e00b4d40 con 0x7f91e0159990 2024-01-16T15:00:12.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.942+0000 7f91d7e00640 1 -- 172.21.15.143:0/2668467430 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f91d807ae20 con 0x7f91e0159990 2024-01-16T15:00:12.924 INFO:teuthology.orchestra.run.smithi143.stdout:55834574855 2024-01-16T15:00:12.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.945+0000 7f91e7000640 1 -- 172.21.15.143:0/2668467430 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f91b806b310 msgr2=0x7f91b806d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.945+0000 7f91e7000640 1 --2- 172.21.15.143:0/2668467430 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f91b806b310 0x7f91b806d7d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f91d0001590 tx=0x7f91d00078f0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.945+0000 7f91e7000640 1 -- 172.21.15.143:0/2668467430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91e0159990 msgr2=0x7f91e00cec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.945+0000 7f91e7000640 1 --2- 172.21.15.143:0/2668467430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91e0159990 0x7f91e00cec90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009a90 tx=0x7f91d8004a40 comp rx=0 tx=0).stop 2024-01-16T15:00:12.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.945+0000 7f91e7000640 1 -- 172.21.15.143:0/2668467430 shutdown_connections 2024-01-16T15:00:12.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.945+0000 7f91e7000640 1 --2- 172.21.15.143:0/2668467430 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f91b806b310 0x7f91b806d7d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.945+0000 7f91e7000640 1 --2- 172.21.15.143:0/2668467430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91e0159990 0x7f91e00cec90 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.945+0000 7f91e7000640 1 --2- 172.21.15.143:0/2668467430 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91e00ce120 0x7f91e015c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.945+0000 7f91e7000640 1 --2- 172.21.15.143:0/2668467430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91e00b1840 0x7f91e0153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.945+0000 7f91e7000640 1 -- 172.21.15.143:0/2668467430 >> 172.21.15.143:0/2668467430 conn(0x7f91e00b4500 msgr2=0x7f91e00cb950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:12.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.945+0000 7f72e4600640 1 -- 172.21.15.143:0/1857962963 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f72dc0b3430 con 0x7f72dc0d1040 2024-01-16T15:00:12.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.945+0000 7f91e7000640 1 -- 172.21.15.143:0/2668467430 shutdown_connections 2024-01-16T15:00:12.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.945+0000 7f91e7000640 1 -- 172.21.15.143:0/2668467430 wait complete. 2024-01-16T15:00:12.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.946+0000 7f72d9600640 1 -- 172.21.15.143:0/1857962963 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f72d4057d70 con 0x7f72dc0d1040 2024-01-16T15:00:12.928 INFO:teuthology.orchestra.run.smithi143.stdout:55834574854 2024-01-16T15:00:12.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.950+0000 7f72e4600640 1 -- 172.21.15.143:0/1857962963 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f72b406b320 msgr2=0x7f72b406d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.950+0000 7f72e4600640 1 --2- 172.21.15.143:0/1857962963 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f72b406b320 0x7f72b406d7e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f72c4035000 tx=0x7f72c4008030 comp rx=0 tx=0).stop 2024-01-16T15:00:12.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.950+0000 7f72e4600640 1 -- 172.21.15.143:0/1857962963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72dc0d1040 msgr2=0x7f72dc1459c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.950+0000 7f72e4600640 1 --2- 172.21.15.143:0/1857962963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72dc0d1040 0x7f72dc1459c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f72d400efc0 tx=0x7f72d400c490 comp rx=0 tx=0).stop 2024-01-16T15:00:12.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.950+0000 7f72e4600640 1 -- 172.21.15.143:0/1857962963 shutdown_connections 2024-01-16T15:00:12.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.950+0000 7f72e4600640 1 --2- 172.21.15.143:0/1857962963 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f72b406b320 0x7f72b406d7e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.950+0000 7f72e4600640 1 --2- 172.21.15.143:0/1857962963 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72dc15d780 0x7f72dc149660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.950+0000 7f72e4600640 1 --2- 172.21.15.143:0/1857962963 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f72dc0d3400 0x7f72dc0c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.950+0000 7f72e4600640 1 --2- 172.21.15.143:0/1857962963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72dc0d1040 0x7f72dc1459c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.950+0000 7f72e4600640 1 -- 172.21.15.143:0/1857962963 >> 172.21.15.143:0/1857962963 conn(0x7f72dc08cec0 msgr2=0x7f72dc0b0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:12.933 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.5 2024-01-16T15:00:12.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.950+0000 7f72e4600640 1 -- 172.21.15.143:0/1857962963 shutdown_connections 2024-01-16T15:00:12.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.951+0000 7f72e4600640 1 -- 172.21.15.143:0/1857962963 wait complete. 2024-01-16T15:00:12.933 DEBUG:teuthology.parallel:result is None 2024-01-16T15:00:12.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.955+0000 7f5c77200640 1 -- 172.21.15.143:0/860947532 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5c7004e840 con 0x7f5c7015d590 2024-01-16T15:00:12.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.955+0000 7f5c6e000640 1 -- 172.21.15.143:0/860947532 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5c68018090 con 0x7f5c7015d590 2024-01-16T15:00:12.936 INFO:teuthology.orchestra.run.smithi143.stdout:55834574856 2024-01-16T15:00:12.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.959+0000 7f5c77200640 1 -- 172.21.15.143:0/860947532 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f5c4806b390 msgr2=0x7f5c4806d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.959+0000 7f5c77200640 1 --2- 172.21.15.143:0/860947532 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f5c4806b390 0x7f5c4806d850 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5c700b88d0 tx=0x7f5c5803c040 comp rx=0 tx=0).stop 2024-01-16T15:00:12.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.959+0000 7f5c77200640 1 -- 172.21.15.143:0/860947532 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c7015d590 msgr2=0x7f5c700b7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.959+0000 7f5c77200640 1 --2- 172.21.15.143:0/860947532 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c7015d590 0x7f5c700b7cb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5c700d3df0 tx=0x7f5c6800ef90 comp rx=0 tx=0).stop 2024-01-16T15:00:12.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.959+0000 7f5c77200640 1 -- 172.21.15.143:0/860947532 shutdown_connections 2024-01-16T15:00:12.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.959+0000 7f5c77200640 1 --2- 172.21.15.143:0/860947532 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f5c4806b390 0x7f5c4806d850 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.959+0000 7f5c77200640 1 --2- 172.21.15.143:0/860947532 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c7015d590 0x7f5c700b7cb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.959+0000 7f5c77200640 1 --2- 172.21.15.143:0/860947532 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c700d3940 0x7f5c70203c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.959+0000 7f5c77200640 1 --2- 172.21.15.143:0/860947532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c700b9000 0x7f5c7006dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.959+0000 7f5c77200640 1 -- 172.21.15.143:0/860947532 >> 172.21.15.143:0/860947532 conn(0x7f5c70061550 msgr2=0x7f5c700cc380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:12.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.959+0000 7f5c77200640 1 -- 172.21.15.143:0/860947532 shutdown_connections 2024-01-16T15:00:12.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.959+0000 7f5c77200640 1 -- 172.21.15.143:0/860947532 wait complete. 2024-01-16T15:00:12.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.962+0000 7fdb18600640 1 -- 172.21.15.143:0/1204674242 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fdb10164b70 con 0x7fdb100cc470 2024-01-16T15:00:12.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.962+0000 7fdb0d600640 1 -- 172.21.15.143:0/1204674242 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdb08054970 con 0x7fdb100cc470 2024-01-16T15:00:12.943 INFO:teuthology.orchestra.run.smithi143.stdout:55834574854 2024-01-16T15:00:12.945 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2024-01-16T15:00:12.946 DEBUG:teuthology.parallel:result is None 2024-01-16T15:00:12.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.965+0000 7fdb18600640 1 -- 172.21.15.143:0/1204674242 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fdae806b310 msgr2=0x7fdae806d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.965+0000 7fdb18600640 1 --2- 172.21.15.143:0/1204674242 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fdae806b310 0x7fdae806d7d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdb10161070 tx=0x7fdb00009040 comp rx=0 tx=0).stop 2024-01-16T15:00:12.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.965+0000 7fdb18600640 1 -- 172.21.15.143:0/1204674242 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb100cc470 msgr2=0x7fdb10160750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.965+0000 7fdb18600640 1 --2- 172.21.15.143:0/1204674242 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb100cc470 0x7fdb10160750 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdb0800c7f0 tx=0x7fdb0800ccb0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.966+0000 7fdb18600640 1 -- 172.21.15.143:0/1204674242 shutdown_connections 2024-01-16T15:00:12.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.966+0000 7fdb18600640 1 --2- 172.21.15.143:0/1204674242 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fdae806b310 0x7fdae806d7d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.966+0000 7fdb18600640 1 --2- 172.21.15.143:0/1204674242 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb100cfb60 0x7fdb1015e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.966+0000 7fdb18600640 1 --2- 172.21.15.143:0/1204674242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb100ce640 0x7fdb10164ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.966+0000 7fdb18600640 1 --2- 172.21.15.143:0/1204674242 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb100cc470 0x7fdb10160750 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.966+0000 7fdb18600640 1 -- 172.21.15.143:0/1204674242 >> 172.21.15.143:0/1204674242 conn(0x7fdb1006a920 msgr2=0x7fdb100ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:12.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.966+0000 7fdb18600640 1 -- 172.21.15.143:0/1204674242 shutdown_connections 2024-01-16T15:00:12.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.966+0000 7fdb18600640 1 -- 172.21.15.143:0/1204674242 wait complete. 2024-01-16T15:00:12.955 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2024-01-16T15:00:12.955 DEBUG:teuthology.parallel:result is None 2024-01-16T15:00:12.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.978+0000 7fb444600640 1 -- 172.21.15.143:0/1918003465 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb43c0ae070 con 0x7fb43c0b0380 2024-01-16T15:00:12.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.979+0000 7fb439600640 1 -- 172.21.15.143:0/1918003465 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb42c034020 con 0x7fb43c0b0380 2024-01-16T15:00:12.960 INFO:teuthology.orchestra.run.smithi143.stdout:55834574853 2024-01-16T15:00:12.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.982+0000 7fb444600640 1 -- 172.21.15.143:0/1918003465 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb41406b2f0 msgr2=0x7fb41406d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.982+0000 7fb444600640 1 --2- 172.21.15.143:0/1918003465 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb41406b2f0 0x7fb41406d7b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb43c0cbb90 tx=0x7fb424009040 comp rx=0 tx=0).stop 2024-01-16T15:00:12.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.982+0000 7fb444600640 1 -- 172.21.15.143:0/1918003465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb43c0b0380 msgr2=0x7fb43c145970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:12.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.982+0000 7fb444600640 1 --2- 172.21.15.143:0/1918003465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb43c0b0380 0x7fb43c145970 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb42c0098f0 tx=0x7fb42c004730 comp rx=0 tx=0).stop 2024-01-16T15:00:12.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.982+0000 7fb444600640 1 -- 172.21.15.143:0/1918003465 shutdown_connections 2024-01-16T15:00:12.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.982+0000 7fb444600640 1 --2- 172.21.15.143:0/1918003465 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb41406b2f0 0x7fb41406d7b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.982+0000 7fb444600640 1 --2- 172.21.15.143:0/1918003465 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb43c0ce120 0x7fb43c167590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.982+0000 7fb444600640 1 --2- 172.21.15.143:0/1918003465 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb43c0b0ce0 0x7fb43c159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.982+0000 7fb444600640 1 --2- 172.21.15.143:0/1918003465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb43c0b0380 0x7fb43c145970 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:12.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.982+0000 7fb444600640 1 -- 172.21.15.143:0/1918003465 >> 172.21.15.143:0/1918003465 conn(0x7fb43c0b4500 msgr2=0x7fb43c0b2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:12.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.982+0000 7fb444600640 1 -- 172.21.15.143:0/1918003465 shutdown_connections 2024-01-16T15:00:12.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:12.982+0000 7fb444600640 1 -- 172.21.15.143:0/1918003465 wait complete. 2024-01-16T15:00:12.965 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.1 2024-01-16T15:00:12.965 DEBUG:teuthology.parallel:result is None 2024-01-16T15:00:12.966 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2024-01-16T15:00:12.966 DEBUG:teuthology.parallel:result is None 2024-01-16T15:00:12.979 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.3 2024-01-16T15:00:12.979 DEBUG:teuthology.parallel:result is None 2024-01-16T15:00:12.980 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-01-16T15:00:12.980 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-16T15:00:13.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.232+0000 7f2098600640 1 Processor -- start 2024-01-16T15:00:13.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.232+0000 7f2098600640 1 -- start start 2024-01-16T15:00:13.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.233+0000 7f2098600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f20900cf450 0x7f20900cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:13.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.233+0000 7f2098600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f20900cdfb0 0x7f20900ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:13.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.233+0000 7f2098600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20900cece0 con 0x7f209015c7f0 2024-01-16T15:00:13.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.234+0000 7f2098600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20900cfe40 con 0x7f20900cdfb0 2024-01-16T15:00:13.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.234+0000 7f2098600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2090156010 con 0x7f20900cf450 2024-01-16T15:00:13.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.234+0000 7f2096600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f20900cdfb0 0x7f20900ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:13.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.234+0000 7f2095c00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f209015c7f0 0x7f209006db20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54206/0 (socket says 172.21.15.143:54206) 2024-01-16T15:00:13.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.234+0000 7f2095c00640 1 -- 172.21.15.143:0/733728156 learned_addr learned my addr 172.21.15.143:0/733728156 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:13.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.234+0000 7f2095200640 1 --2- 172.21.15.143:0/733728156 >> v2:172.21.15.204:3300/0 conn(0x7f20900cf450 0x7f20900cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:13.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.234+0000 7f2096600640 1 -- 172.21.15.143:0/733728156 >> v2:172.21.15.204:3300/0 conn(0x7f20900cf450 msgr2=0x7f20900cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:13.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.234+0000 7f2096600640 1 --2- 172.21.15.143:0/733728156 >> v2:172.21.15.204:3300/0 conn(0x7f20900cf450 0x7f20900cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:13.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.234+0000 7f2096600640 1 -- 172.21.15.143:0/733728156 >> v1:172.21.15.143:6789/0 conn(0x7f209015c7f0 legacy=0x7f209006db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:13.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.234+0000 7f208fe00640 1 -- 172.21.15.143:0/733728156 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3278395306 0 0) 0x7f20900cece0 con 0x7f209015c7f0 2024-01-16T15:00:13.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.234+0000 7f2096600640 1 -- 172.21.15.143:0/733728156 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f209006d040 con 0x7f20900cdfb0 2024-01-16T15:00:13.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.235+0000 7f2096600640 1 --2- 172.21.15.143:0/733728156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20900cdfb0 0x7f20900ce3e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2090070410 tx=0x7f208802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c809f512eb5315c server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:13.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.236+0000 7f208fe00640 1 -- 172.21.15.143:0/733728156 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208802cc80 con 0x7f20900cdfb0 2024-01-16T15:00:13.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.236+0000 7f208fe00640 1 -- 172.21.15.143:0/733728156 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f208802cde0 con 0x7f20900cdfb0 2024-01-16T15:00:13.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.236+0000 7f208fe00640 1 -- 172.21.15.143:0/733728156 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2088034660 con 0x7f20900cdfb0 2024-01-16T15:00:13.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.236+0000 7f2098600640 1 -- 172.21.15.143:0/733728156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20900cdfb0 msgr2=0x7f20900ce3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:13.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.236+0000 7f2098600640 1 --2- 172.21.15.143:0/733728156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20900cdfb0 0x7f20900ce3e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2090070410 tx=0x7f208802bed0 comp rx=0 tx=0).stop 2024-01-16T15:00:13.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.236+0000 7f2098600640 1 -- 172.21.15.143:0/733728156 shutdown_connections 2024-01-16T15:00:13.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.236+0000 7f2098600640 1 --2- 172.21.15.143:0/733728156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20900cdfb0 0x7f20900ce3e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:13.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.236+0000 7f2098600640 1 --2- 172.21.15.143:0/733728156 >> v2:172.21.15.204:3300/0 conn(0x7f20900cf450 0x7f20900cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:13.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.236+0000 7f2098600640 1 -- 172.21.15.143:0/733728156 >> 172.21.15.143:0/733728156 conn(0x7f209006a920 msgr2=0x7f209006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:13.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.236+0000 7f2098600640 1 -- 172.21.15.143:0/733728156 shutdown_connections 2024-01-16T15:00:13.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.236+0000 7f2098600640 1 -- 172.21.15.143:0/733728156 wait complete. 2024-01-16T15:00:13.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.238+0000 7f2098600640 1 Processor -- start 2024-01-16T15:00:13.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.238+0000 7f2098600640 1 -- start start 2024-01-16T15:00:13.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.239+0000 7f2098600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20900cdfb0 0x7f2090152f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:13.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.239+0000 7f2098600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20900cf450 0x7f2090158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:13.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.239+0000 7f2098600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f209015c7f0 0x7f209015ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:13.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.239+0000 7f2098600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209015a5d0 con 0x7f20900cf450 2024-01-16T15:00:13.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.239+0000 7f2098600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2090151af0 con 0x7f20900cdfb0 2024-01-16T15:00:13.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.239+0000 7f2098600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2090151270 con 0x7f209015c7f0 2024-01-16T15:00:13.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.239+0000 7f2095200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20900cf450 0x7f2090158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:13.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.239+0000 7f2095200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20900cf450 0x7f2090158bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42892/0 (socket says 172.21.15.143:42892) 2024-01-16T15:00:13.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.239+0000 7f2095200640 1 -- 172.21.15.143:0/2495149187 learned_addr learned my addr 172.21.15.143:0/2495149187 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:13.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.240+0000 7f2095c00640 1 --2- 172.21.15.143:0/2495149187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20900cdfb0 0x7f2090152f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:13.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.240+0000 7f2096600640 1 --2- 172.21.15.143:0/2495149187 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f209015c7f0 0x7f209015ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:13.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.240+0000 7f2095200640 1 -- 172.21.15.143:0/2495149187 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f209015c7f0 msgr2=0x7f209015ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:13.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.240+0000 7f2095200640 1 --2- 172.21.15.143:0/2495149187 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f209015c7f0 0x7f209015ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:13.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.240+0000 7f2095200640 1 -- 172.21.15.143:0/2495149187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20900cdfb0 msgr2=0x7f2090152f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:13.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.240+0000 7f2095200640 1 --2- 172.21.15.143:0/2495149187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20900cdfb0 0x7f2090152f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:13.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.240+0000 7f2095200640 1 -- 172.21.15.143:0/2495149187 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2088009a70 con 0x7f20900cf450 2024-01-16T15:00:13.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.240+0000 7f2095c00640 1 --2- 172.21.15.143:0/2495149187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20900cdfb0 0x7f2090152f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:13.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.240+0000 7f2095200640 1 --2- 172.21.15.143:0/2495149187 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20900cf450 0x7f2090158bf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f207800d800 tx=0x7f207800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:13.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.240+0000 7f208d600640 1 -- 172.21.15.143:0/2495149187 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207800b6c0 con 0x7f20900cf450 2024-01-16T15:00:13.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.240+0000 7f208d600640 1 -- 172.21.15.143:0/2495149187 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f207800b820 con 0x7f20900cf450 2024-01-16T15:00:13.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.240+0000 7f2098600640 1 -- 172.21.15.143:0/2495149187 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2090151530 con 0x7f20900cf450 2024-01-16T15:00:13.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.240+0000 7f208d600640 1 -- 172.21.15.143:0/2495149187 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207800bbb0 con 0x7f20900cf450 2024-01-16T15:00:13.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.240+0000 7f2098600640 1 -- 172.21.15.143:0/2495149187 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f209015a260 con 0x7f20900cf450 2024-01-16T15:00:13.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.242+0000 7f2098600640 1 -- 172.21.15.143:0/2495149187 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f209015a780 con 0x7f20900cf450 2024-01-16T15:00:13.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.242+0000 7f208d600640 1 -- 172.21.15.143:0/2495149187 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f207800bd10 con 0x7f20900cf450 2024-01-16T15:00:13.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.243+0000 7f208d600640 1 --2- 172.21.15.143:0/2495149187 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f206806b2f0 0x7f206806d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:13.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.243+0000 7f208d600640 1 -- 172.21.15.143:0/2495149187 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f207808a080 con 0x7f20900cf450 2024-01-16T15:00:13.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.243+0000 7f2095c00640 1 --2- 172.21.15.143:0/2495149187 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f206806b2f0 0x7f206806d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:13.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.244+0000 7f2095c00640 1 --2- 172.21.15.143:0/2495149187 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f206806b2f0 0x7f206806d7b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2080000f60 tx=0x7f2080009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:13.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.247+0000 7f208d600640 1 -- 172.21.15.143:0/2495149187 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f209015a780 con 0x7f20900cf450 2024-01-16T15:00:13.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.422+0000 7f2098600640 1 -- 172.21.15.143:0/2495149187 --> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2090070af0 con 0x7f206806b2f0 2024-01-16T15:00:13.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.435+0000 7f208d600640 1 -- 172.21.15.143:0/2495149187 <== mgr.4113 v2:172.21.15.143:6832/3153492115 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264462 (secure 0 0 0) 0x7f2090070af0 con 0x7f206806b2f0 2024-01-16T15:00:13.420 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:00:13.421 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-16T15:00:13.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.441+0000 7f2098600640 1 -- 172.21.15.143:0/2495149187 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f206806b2f0 msgr2=0x7f206806d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:13.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.441+0000 7f2098600640 1 --2- 172.21.15.143:0/2495149187 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f206806b2f0 0x7f206806d7b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2080000f60 tx=0x7f2080009040 comp rx=0 tx=0).stop 2024-01-16T15:00:13.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.441+0000 7f2098600640 1 -- 172.21.15.143:0/2495149187 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20900cf450 msgr2=0x7f2090158bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:13.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.441+0000 7f2098600640 1 --2- 172.21.15.143:0/2495149187 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20900cf450 0x7f2090158bf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f207800d800 tx=0x7f207800dcc0 comp rx=0 tx=0).stop 2024-01-16T15:00:13.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.441+0000 7f2098600640 1 -- 172.21.15.143:0/2495149187 shutdown_connections 2024-01-16T15:00:13.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.441+0000 7f2098600640 1 --2- 172.21.15.143:0/2495149187 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f206806b2f0 0x7f206806d7b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:13.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.441+0000 7f2098600640 1 --2- 172.21.15.143:0/2495149187 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f209015c7f0 0x7f209015ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:13.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.441+0000 7f2098600640 1 --2- 172.21.15.143:0/2495149187 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20900cf450 0x7f2090158bf0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:13.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.441+0000 7f2098600640 1 --2- 172.21.15.143:0/2495149187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20900cdfb0 0x7f2090152f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:13.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.441+0000 7f2098600640 1 -- 172.21.15.143:0/2495149187 >> 172.21.15.143:0/2495149187 conn(0x7f209006a920 msgr2=0x7f209006d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:13.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.441+0000 7f2098600640 1 -- 172.21.15.143:0/2495149187 shutdown_connections 2024-01-16T15:00:13.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.441+0000 7f2098600640 1 -- 172.21.15.143:0/2495149187 wait complete. 2024-01-16T15:00:13.440 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-01-16T15:00:11.654878+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219004,"kb_used_data":2820,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755716,"statfs":{"total":773094113280,"available":772869853184,"internally_reserved":0,"allocated":2887680,"data_stored":1751817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.109770"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.558033+0000","last_change":"2024-01-16T15:00:03.579227+0000","last_active":"2024-01-16T15:00:04.558033+0000","last_peered":"2024-01-16T15:00:04.558033+0000","last_clean":"2024-01-16T15:00:04.558033+0000","last_became_active":"2024-01-16T15:00:03.578921+0000","last_became_peered":"2024-01-16T15:00:03.578921+0000","last_unstale":"2024-01-16T15:00:04.558033+0000","last_undegraded":"2024-01-16T15:00:04.558033+0000","last_fullsized":"2024-01-16T15:00:04.558033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:32:18.387639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.554616+0000","last_change":"2024-01-16T15:00:03.571130+0000","last_active":"2024-01-16T15:00:04.554616+0000","last_peered":"2024-01-16T15:00:04.554616+0000","last_clean":"2024-01-16T15:00:04.554616+0000","last_became_active":"2024-01-16T15:00:03.570816+0000","last_became_peered":"2024-01-16T15:00:03.570816+0000","last_unstale":"2024-01-16T15:00:04.554616+0000","last_undegraded":"2024-01-16T15:00:04.554616+0000","last_fullsized":"2024-01-16T15:00:04.554616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:06:24.858403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323971+0000","last_change":"2024-01-16T15:00:03.578171+0000","last_active":"2024-01-16T15:00:05.323971+0000","last_peered":"2024-01-16T15:00:05.323971+0000","last_clean":"2024-01-16T15:00:05.323971+0000","last_became_active":"2024-01-16T15:00:03.577849+0000","last_became_peered":"2024-01-16T15:00:03.577849+0000","last_unstale":"2024-01-16T15:00:05.323971+0000","last_undegraded":"2024-01-16T15:00:05.323971+0000","last_fullsized":"2024-01-16T15:00:05.323971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:11:11.328141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323719+0000","last_change":"2024-01-16T15:00:03.585019+0000","last_active":"2024-01-16T15:00:05.323719+0000","last_peered":"2024-01-16T15:00:05.323719+0000","last_clean":"2024-01-16T15:00:05.323719+0000","last_became_active":"2024-01-16T15:00:03.584747+0000","last_became_peered":"2024-01-16T15:00:03.584747+0000","last_unstale":"2024-01-16T15:00:05.323719+0000","last_undegraded":"2024-01-16T15:00:05.323719+0000","last_fullsized":"2024-01-16T15:00:05.323719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:44:37.235476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.556701+0000","last_change":"2024-01-16T15:00:03.577404+0000","last_active":"2024-01-16T15:00:04.556701+0000","last_peered":"2024-01-16T15:00:04.556701+0000","last_clean":"2024-01-16T15:00:04.556701+0000","last_became_active":"2024-01-16T15:00:03.577051+0000","last_became_peered":"2024-01-16T15:00:03.577051+0000","last_unstale":"2024-01-16T15:00:04.556701+0000","last_undegraded":"2024-01-16T15:00:04.556701+0000","last_fullsized":"2024-01-16T15:00:04.556701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:44:37.235476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.322548+0000","last_change":"2024-01-16T15:00:03.589065+0000","last_active":"2024-01-16T15:00:05.322548+0000","last_peered":"2024-01-16T15:00:05.322548+0000","last_clean":"2024-01-16T15:00:05.322548+0000","last_became_active":"2024-01-16T15:00:03.588776+0000","last_became_peered":"2024-01-16T15:00:03.588776+0000","last_unstale":"2024-01-16T15:00:05.322548+0000","last_undegraded":"2024-01-16T15:00:05.322548+0000","last_fullsized":"2024-01-16T15:00:05.322548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:06:24.858403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.556141+0000","last_change":"2024-01-16T15:00:03.570611+0000","last_active":"2024-01-16T15:00:04.556141+0000","last_peered":"2024-01-16T15:00:04.556141+0000","last_clean":"2024-01-16T15:00:04.556141+0000","last_became_active":"2024-01-16T15:00:03.569839+0000","last_became_peered":"2024-01-16T15:00:03.569839+0000","last_unstale":"2024-01-16T15:00:04.556141+0000","last_undegraded":"2024-01-16T15:00:04.556141+0000","last_fullsized":"2024-01-16T15:00:04.556141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:09:47.428018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.127040+0000","last_change":"2024-01-16T15:00:03.586954+0000","last_active":"2024-01-16T15:00:05.127040+0000","last_peered":"2024-01-16T15:00:05.127040+0000","last_clean":"2024-01-16T15:00:05.127040+0000","last_became_active":"2024-01-16T15:00:03.586663+0000","last_became_peered":"2024-01-16T15:00:03.586663+0000","last_unstale":"2024-01-16T15:00:05.127040+0000","last_undegraded":"2024-01-16T15:00:05.127040+0000","last_fullsized":"2024-01-16T15:00:05.127040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:34:39.229613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.558126+0000","last_change":"2024-01-16T15:00:03.596120+0000","last_active":"2024-01-16T15:00:04.558126+0000","last_peered":"2024-01-16T15:00:04.558126+0000","last_clean":"2024-01-16T15:00:04.558126+0000","last_became_active":"2024-01-16T15:00:03.595866+0000","last_became_peered":"2024-01-16T15:00:03.595866+0000","last_unstale":"2024-01-16T15:00:04.558126+0000","last_undegraded":"2024-01-16T15:00:04.558126+0000","last_fullsized":"2024-01-16T15:00:04.558126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:53:57.952288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.750726+0000","last_change":"2024-01-16T15:00:03.588019+0000","last_active":"2024-01-16T15:00:04.750726+0000","last_peered":"2024-01-16T15:00:04.750726+0000","last_clean":"2024-01-16T15:00:04.750726+0000","last_became_active":"2024-01-16T15:00:03.587359+0000","last_became_peered":"2024-01-16T15:00:03.587359+0000","last_unstale":"2024-01-16T15:00:04.750726+0000","last_undegraded":"2024-01-16T15:00:04.750726+0000","last_fullsized":"2024-01-16T15:00:04.750726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:31:56.335251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.750616+0000","last_change":"2024-01-16T15:00:03.590950+0000","last_active":"2024-01-16T15:00:04.750616+0000","last_peered":"2024-01-16T15:00:04.750616+0000","last_clean":"2024-01-16T15:00:04.750616+0000","last_became_active":"2024-01-16T15:00:03.590702+0000","last_became_peered":"2024-01-16T15:00:03.590702+0000","last_unstale":"2024-01-16T15:00:04.750616+0000","last_undegraded":"2024-01-16T15:00:04.750616+0000","last_fullsized":"2024-01-16T15:00:04.750616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:04:13.308774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.555500+0000","last_change":"2024-01-16T15:00:03.582929+0000","last_active":"2024-01-16T15:00:04.555500+0000","last_peered":"2024-01-16T15:00:04.555500+0000","last_clean":"2024-01-16T15:00:04.555500+0000","last_became_active":"2024-01-16T15:00:03.582040+0000","last_became_peered":"2024-01-16T15:00:03.582040+0000","last_unstale":"2024-01-16T15:00:04.555500+0000","last_undegraded":"2024-01-16T15:00:04.555500+0000","last_fullsized":"2024-01-16T15:00:04.555500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:10:55.291745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.127726+0000","last_change":"2024-01-16T15:00:03.581183+0000","last_active":"2024-01-16T15:00:05.127726+0000","last_peered":"2024-01-16T15:00:05.127726+0000","last_clean":"2024-01-16T15:00:05.127726+0000","last_became_active":"2024-01-16T15:00:03.578642+0000","last_became_peered":"2024-01-16T15:00:03.578642+0000","last_unstale":"2024-01-16T15:00:05.127726+0000","last_undegraded":"2024-01-16T15:00:05.127726+0000","last_fullsized":"2024-01-16T15:00:05.127726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:06:24.858403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323409+0000","last_change":"2024-01-16T15:00:03.570282+0000","last_active":"2024-01-16T15:00:05.323409+0000","last_peered":"2024-01-16T15:00:05.323409+0000","last_clean":"2024-01-16T15:00:05.323409+0000","last_became_active":"2024-01-16T15:00:03.569988+0000","last_became_peered":"2024-01-16T15:00:03.569988+0000","last_unstale":"2024-01-16T15:00:05.323409+0000","last_undegraded":"2024-01-16T15:00:05.323409+0000","last_fullsized":"2024-01-16T15:00:05.323409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:16:00.005822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.554056+0000","last_change":"2024-01-16T15:00:03.584705+0000","last_active":"2024-01-16T15:00:04.554056+0000","last_peered":"2024-01-16T15:00:04.554056+0000","last_clean":"2024-01-16T15:00:04.554056+0000","last_became_active":"2024-01-16T15:00:03.584361+0000","last_became_peered":"2024-01-16T15:00:03.584361+0000","last_unstale":"2024-01-16T15:00:04.554056+0000","last_undegraded":"2024-01-16T15:00:04.554056+0000","last_fullsized":"2024-01-16T15:00:04.554056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:11:11.328141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.322398+0000","last_change":"2024-01-16T15:00:03.587177+0000","last_active":"2024-01-16T15:00:05.322398+0000","last_peered":"2024-01-16T15:00:05.322398+0000","last_clean":"2024-01-16T15:00:05.322398+0000","last_became_active":"2024-01-16T15:00:03.586803+0000","last_became_peered":"2024-01-16T15:00:03.586803+0000","last_unstale":"2024-01-16T15:00:05.322398+0000","last_undegraded":"2024-01-16T15:00:05.322398+0000","last_fullsized":"2024-01-16T15:00:05.322398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:03:10.226190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323936+0000","last_change":"2024-01-16T15:00:03.595310+0000","last_active":"2024-01-16T15:00:05.323936+0000","last_peered":"2024-01-16T15:00:05.323936+0000","last_clean":"2024-01-16T15:00:05.323936+0000","last_became_active":"2024-01-16T15:00:03.589586+0000","last_became_peered":"2024-01-16T15:00:03.589586+0000","last_unstale":"2024-01-16T15:00:05.323936+0000","last_undegraded":"2024-01-16T15:00:05.323936+0000","last_fullsized":"2024-01-16T15:00:05.323936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:29:26.138536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.322465+0000","last_change":"2024-01-16T15:00:03.583181+0000","last_active":"2024-01-16T15:00:05.322465+0000","last_peered":"2024-01-16T15:00:05.322465+0000","last_clean":"2024-01-16T15:00:05.322465+0000","last_became_active":"2024-01-16T15:00:03.582404+0000","last_became_peered":"2024-01-16T15:00:03.582404+0000","last_unstale":"2024-01-16T15:00:05.322465+0000","last_undegraded":"2024-01-16T15:00:05.322465+0000","last_fullsized":"2024-01-16T15:00:05.322465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:25:28.863720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.557296+0000","last_change":"2024-01-16T15:00:03.577376+0000","last_active":"2024-01-16T15:00:04.557296+0000","last_peered":"2024-01-16T15:00:04.557296+0000","last_clean":"2024-01-16T15:00:04.557296+0000","last_became_active":"2024-01-16T15:00:03.577167+0000","last_became_peered":"2024-01-16T15:00:03.577167+0000","last_unstale":"2024-01-16T15:00:04.557296+0000","last_undegraded":"2024-01-16T15:00:04.557296+0000","last_fullsized":"2024-01-16T15:00:04.557296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:14:46.103689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.750577+0000","last_change":"2024-01-16T15:00:03.587122+0000","last_active":"2024-01-16T15:00:04.750577+0000","last_peered":"2024-01-16T15:00:04.750577+0000","last_clean":"2024-01-16T15:00:04.750577+0000","last_became_active":"2024-01-16T15:00:03.586244+0000","last_became_peered":"2024-01-16T15:00:03.586244+0000","last_unstale":"2024-01-16T15:00:04.750577+0000","last_undegraded":"2024-01-16T15:00:04.750577+0000","last_fullsized":"2024-01-16T15:00:04.750577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:57:14.928029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323320+0000","last_change":"2024-01-16T15:00:03.572186+0000","last_active":"2024-01-16T15:00:05.323320+0000","last_peered":"2024-01-16T15:00:05.323320+0000","last_clean":"2024-01-16T15:00:05.323320+0000","last_became_active":"2024-01-16T15:00:03.571795+0000","last_became_peered":"2024-01-16T15:00:03.571795+0000","last_unstale":"2024-01-16T15:00:05.323320+0000","last_undegraded":"2024-01-16T15:00:05.323320+0000","last_fullsized":"2024-01-16T15:00:05.323320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:18:11.185813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.322924+0000","last_change":"2024-01-16T15:00:03.589661+0000","last_active":"2024-01-16T15:00:05.322924+0000","last_peered":"2024-01-16T15:00:05.322924+0000","last_clean":"2024-01-16T15:00:05.322924+0000","last_became_active":"2024-01-16T15:00:03.589365+0000","last_became_peered":"2024-01-16T15:00:03.589365+0000","last_unstale":"2024-01-16T15:00:05.322924+0000","last_undegraded":"2024-01-16T15:00:05.322924+0000","last_fullsized":"2024-01-16T15:00:05.322924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:04:13.308774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.127047+0000","last_change":"2024-01-16T15:00:03.591522+0000","last_active":"2024-01-16T15:00:05.127047+0000","last_peered":"2024-01-16T15:00:05.127047+0000","last_clean":"2024-01-16T15:00:05.127047+0000","last_became_active":"2024-01-16T15:00:03.591143+0000","last_became_peered":"2024-01-16T15:00:03.591143+0000","last_unstale":"2024-01-16T15:00:05.127047+0000","last_undegraded":"2024-01-16T15:00:05.127047+0000","last_fullsized":"2024-01-16T15:00:05.127047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:11:11.328141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.322833+0000","last_change":"2024-01-16T15:00:03.585381+0000","last_active":"2024-01-16T15:00:05.322833+0000","last_peered":"2024-01-16T15:00:05.322833+0000","last_clean":"2024-01-16T15:00:05.322833+0000","last_became_active":"2024-01-16T15:00:03.584858+0000","last_became_peered":"2024-01-16T15:00:03.584858+0000","last_unstale":"2024-01-16T15:00:05.322833+0000","last_undegraded":"2024-01-16T15:00:05.322833+0000","last_fullsized":"2024-01-16T15:00:05.322833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:57:14.928029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.127981+0000","last_change":"2024-01-16T15:00:03.591627+0000","last_active":"2024-01-16T15:00:05.127981+0000","last_peered":"2024-01-16T15:00:05.127981+0000","last_clean":"2024-01-16T15:00:05.127981+0000","last_became_active":"2024-01-16T15:00:03.591216+0000","last_became_peered":"2024-01-16T15:00:03.591216+0000","last_unstale":"2024-01-16T15:00:05.127981+0000","last_undegraded":"2024-01-16T15:00:05.127981+0000","last_fullsized":"2024-01-16T15:00:05.127981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:31:56.335251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.750822+0000","last_change":"2024-01-16T15:00:03.588262+0000","last_active":"2024-01-16T15:00:04.750822+0000","last_peered":"2024-01-16T15:00:04.750822+0000","last_clean":"2024-01-16T15:00:04.750822+0000","last_became_active":"2024-01-16T15:00:03.587728+0000","last_became_peered":"2024-01-16T15:00:03.587728+0000","last_unstale":"2024-01-16T15:00:04.750822+0000","last_undegraded":"2024-01-16T15:00:04.750822+0000","last_fullsized":"2024-01-16T15:00:04.750822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:05:08.127179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.554267+0000","last_change":"2024-01-16T15:00:03.576892+0000","last_active":"2024-01-16T15:00:04.554267+0000","last_peered":"2024-01-16T15:00:04.554267+0000","last_clean":"2024-01-16T15:00:04.554267+0000","last_became_active":"2024-01-16T15:00:03.574566+0000","last_became_peered":"2024-01-16T15:00:03.574566+0000","last_unstale":"2024-01-16T15:00:04.554267+0000","last_undegraded":"2024-01-16T15:00:04.554267+0000","last_fullsized":"2024-01-16T15:00:04.554267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:33:59.253631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.127898+0000","last_change":"2024-01-16T15:00:03.591601+0000","last_active":"2024-01-16T15:00:05.127898+0000","last_peered":"2024-01-16T15:00:05.127898+0000","last_clean":"2024-01-16T15:00:05.127898+0000","last_became_active":"2024-01-16T15:00:03.591271+0000","last_became_peered":"2024-01-16T15:00:03.591271+0000","last_unstale":"2024-01-16T15:00:05.127898+0000","last_undegraded":"2024-01-16T15:00:05.127898+0000","last_fullsized":"2024-01-16T15:00:05.127898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:04:13.308774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323503+0000","last_change":"2024-01-16T15:00:03.580655+0000","last_active":"2024-01-16T15:00:05.323503+0000","last_peered":"2024-01-16T15:00:05.323503+0000","last_clean":"2024-01-16T15:00:05.323503+0000","last_became_active":"2024-01-16T15:00:03.580091+0000","last_became_peered":"2024-01-16T15:00:03.580091+0000","last_unstale":"2024-01-16T15:00:05.323503+0000","last_undegraded":"2024-01-16T15:00:05.323503+0000","last_fullsized":"2024-01-16T15:00:05.323503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:34:39.229613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.322959+0000","last_change":"2024-01-16T15:00:03.588486+0000","last_active":"2024-01-16T15:00:05.322959+0000","last_peered":"2024-01-16T15:00:05.322959+0000","last_clean":"2024-01-16T15:00:05.322959+0000","last_became_active":"2024-01-16T15:00:03.586447+0000","last_became_peered":"2024-01-16T15:00:03.586447+0000","last_unstale":"2024-01-16T15:00:05.322959+0000","last_undegraded":"2024-01-16T15:00:05.322959+0000","last_fullsized":"2024-01-16T15:00:05.322959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:31:56.335251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.322016+0000","last_change":"2024-01-16T15:00:03.581151+0000","last_active":"2024-01-16T15:00:05.322016+0000","last_peered":"2024-01-16T15:00:05.322016+0000","last_clean":"2024-01-16T15:00:05.322016+0000","last_became_active":"2024-01-16T15:00:03.579463+0000","last_became_peered":"2024-01-16T15:00:03.579463+0000","last_unstale":"2024-01-16T15:00:05.322016+0000","last_undegraded":"2024-01-16T15:00:05.322016+0000","last_fullsized":"2024-01-16T15:00:05.322016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:10:55.291745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.750064+0000","last_change":"2024-01-16T15:00:03.588682+0000","last_active":"2024-01-16T15:00:04.750064+0000","last_peered":"2024-01-16T15:00:04.750064+0000","last_clean":"2024-01-16T15:00:04.750064+0000","last_became_active":"2024-01-16T15:00:03.588148+0000","last_became_peered":"2024-01-16T15:00:03.588148+0000","last_unstale":"2024-01-16T15:00:04.750064+0000","last_undegraded":"2024-01-16T15:00:04.750064+0000","last_fullsized":"2024-01-16T15:00:04.750064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:33:59.253631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.555160+0000","last_change":"2024-01-16T15:00:03.580412+0000","last_active":"2024-01-16T15:00:04.555160+0000","last_peered":"2024-01-16T15:00:04.555160+0000","last_clean":"2024-01-16T15:00:04.555160+0000","last_became_active":"2024-01-16T15:00:03.572852+0000","last_became_peered":"2024-01-16T15:00:03.572852+0000","last_unstale":"2024-01-16T15:00:04.555160+0000","last_undegraded":"2024-01-16T15:00:04.555160+0000","last_fullsized":"2024-01-16T15:00:04.555160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:48:44.736531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.613598+0000","last_change":"2024-01-16T15:00:02.635074+0000","last_active":"2024-01-16T15:00:03.613598+0000","last_peered":"2024-01-16T15:00:03.613598+0000","last_clean":"2024-01-16T15:00:03.613598+0000","last_became_active":"2024-01-16T15:00:02.634767+0000","last_became_peered":"2024-01-16T15:00:02.634767+0000","last_unstale":"2024-01-16T15:00:03.613598+0000","last_undegraded":"2024-01-16T15:00:03.613598+0000","last_fullsized":"2024-01-16T15:00:03.613598+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:15:58.991139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.324292+0000","last_change":"2024-01-16T15:00:03.573187+0000","last_active":"2024-01-16T15:00:05.324292+0000","last_peered":"2024-01-16T15:00:05.324292+0000","last_clean":"2024-01-16T15:00:05.324292+0000","last_became_active":"2024-01-16T15:00:03.572755+0000","last_became_peered":"2024-01-16T15:00:03.572755+0000","last_unstale":"2024-01-16T15:00:05.324292+0000","last_undegraded":"2024-01-16T15:00:05.324292+0000","last_fullsized":"2024-01-16T15:00:05.324292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:06:24.858403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.610883+0000","last_change":"2024-01-16T15:00:02.632028+0000","last_active":"2024-01-16T15:00:03.610883+0000","last_peered":"2024-01-16T15:00:03.610883+0000","last_clean":"2024-01-16T15:00:03.610883+0000","last_became_active":"2024-01-16T15:00:02.626019+0000","last_became_peered":"2024-01-16T15:00:02.626019+0000","last_unstale":"2024-01-16T15:00:03.610883+0000","last_undegraded":"2024-01-16T15:00:03.610883+0000","last_fullsized":"2024-01-16T15:00:03.610883+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:48:43.721848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.568172+0000","last_change":"2024-01-16T15:00:02.635363+0000","last_active":"2024-01-16T15:00:03.568172+0000","last_peered":"2024-01-16T15:00:03.568172+0000","last_clean":"2024-01-16T15:00:03.568172+0000","last_became_active":"2024-01-16T15:00:02.634387+0000","last_became_peered":"2024-01-16T15:00:02.634387+0000","last_unstale":"2024-01-16T15:00:03.568172+0000","last_undegraded":"2024-01-16T15:00:03.568172+0000","last_fullsized":"2024-01-16T15:00:03.568172+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:10:54.277062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.127245+0000","last_change":"2024-01-16T15:00:03.576575+0000","last_active":"2024-01-16T15:00:05.127245+0000","last_peered":"2024-01-16T15:00:05.127245+0000","last_clean":"2024-01-16T15:00:05.127245+0000","last_became_active":"2024-01-16T15:00:03.576266+0000","last_became_peered":"2024-01-16T15:00:03.576266+0000","last_unstale":"2024-01-16T15:00:05.127245+0000","last_undegraded":"2024-01-16T15:00:05.127245+0000","last_fullsized":"2024-01-16T15:00:05.127245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:28:03.212278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.584305+0000","last_change":"2024-01-16T15:00:02.636608+0000","last_active":"2024-01-16T15:00:03.584305+0000","last_peered":"2024-01-16T15:00:03.584305+0000","last_clean":"2024-01-16T15:00:03.584305+0000","last_became_active":"2024-01-16T15:00:02.636235+0000","last_became_peered":"2024-01-16T15:00:02.636235+0000","last_unstale":"2024-01-16T15:00:03.584305+0000","last_undegraded":"2024-01-16T15:00:03.584305+0000","last_fullsized":"2024-01-16T15:00:03.584305+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:11:10.313458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.553948+0000","last_change":"2024-01-16T15:00:03.579753+0000","last_active":"2024-01-16T15:00:04.553948+0000","last_peered":"2024-01-16T15:00:04.553948+0000","last_clean":"2024-01-16T15:00:04.553948+0000","last_became_active":"2024-01-16T15:00:03.579415+0000","last_became_peered":"2024-01-16T15:00:03.579415+0000","last_unstale":"2024-01-16T15:00:04.553948+0000","last_undegraded":"2024-01-16T15:00:04.553948+0000","last_fullsized":"2024-01-16T15:00:04.553948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:34:39.229613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.572357+0000","last_change":"2024-01-16T15:00:02.635549+0000","last_active":"2024-01-16T15:00:03.572357+0000","last_peered":"2024-01-16T15:00:03.572357+0000","last_clean":"2024-01-16T15:00:03.572357+0000","last_became_active":"2024-01-16T15:00:02.635309+0000","last_became_peered":"2024-01-16T15:00:02.635309+0000","last_unstale":"2024-01-16T15:00:03.572357+0000","last_undegraded":"2024-01-16T15:00:03.572357+0000","last_fullsized":"2024-01-16T15:00:03.572357+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:34:38.214930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323817+0000","last_change":"2024-01-16T15:00:03.586931+0000","last_active":"2024-01-16T15:00:05.323817+0000","last_peered":"2024-01-16T15:00:05.323817+0000","last_clean":"2024-01-16T15:00:05.323817+0000","last_became_active":"2024-01-16T15:00:03.586531+0000","last_became_peered":"2024-01-16T15:00:03.586531+0000","last_unstale":"2024-01-16T15:00:05.323817+0000","last_undegraded":"2024-01-16T15:00:05.323817+0000","last_fullsized":"2024-01-16T15:00:05.323817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:18:16.333187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.583406+0000","last_change":"2024-01-16T15:00:02.638637+0000","last_active":"2024-01-16T15:00:03.583406+0000","last_peered":"2024-01-16T15:00:03.583406+0000","last_clean":"2024-01-16T15:00:03.583406+0000","last_became_active":"2024-01-16T15:00:02.638355+0000","last_became_peered":"2024-01-16T15:00:02.638355+0000","last_unstale":"2024-01-16T15:00:03.583406+0000","last_undegraded":"2024-01-16T15:00:03.583406+0000","last_fullsized":"2024-01-16T15:00:03.583406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:53:48.731286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.324191+0000","last_change":"2024-01-16T15:00:03.587105+0000","last_active":"2024-01-16T15:00:05.324191+0000","last_peered":"2024-01-16T15:00:05.324191+0000","last_clean":"2024-01-16T15:00:05.324191+0000","last_became_active":"2024-01-16T15:00:03.586572+0000","last_became_peered":"2024-01-16T15:00:03.586572+0000","last_unstale":"2024-01-16T15:00:05.324191+0000","last_undegraded":"2024-01-16T15:00:05.324191+0000","last_fullsized":"2024-01-16T15:00:05.324191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:03:10.226190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.611665+0000","last_change":"2024-01-16T15:00:02.636352+0000","last_active":"2024-01-16T15:00:03.611665+0000","last_peered":"2024-01-16T15:00:03.611665+0000","last_clean":"2024-01-16T15:00:03.611665+0000","last_became_active":"2024-01-16T15:00:02.635069+0000","last_became_peered":"2024-01-16T15:00:02.635069+0000","last_unstale":"2024-01-16T15:00:03.611665+0000","last_undegraded":"2024-01-16T15:00:03.611665+0000","last_fullsized":"2024-01-16T15:00:03.611665+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:10:54.277062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.555540+0000","last_change":"2024-01-16T15:00:03.580722+0000","last_active":"2024-01-16T15:00:04.555540+0000","last_peered":"2024-01-16T15:00:04.555540+0000","last_clean":"2024-01-16T15:00:04.555540+0000","last_became_active":"2024-01-16T15:00:03.572307+0000","last_became_peered":"2024-01-16T15:00:03.572307+0000","last_unstale":"2024-01-16T15:00:04.555540+0000","last_undegraded":"2024-01-16T15:00:04.555540+0000","last_fullsized":"2024-01-16T15:00:04.555540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:25:28.863720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.617703+0000","last_change":"2024-01-16T15:00:02.625026+0000","last_active":"2024-01-16T15:00:03.617703+0000","last_peered":"2024-01-16T15:00:03.617703+0000","last_clean":"2024-01-16T15:00:03.617703+0000","last_became_active":"2024-01-16T15:00:02.622889+0000","last_became_peered":"2024-01-16T15:00:02.622889+0000","last_unstale":"2024-01-16T15:00:03.617703+0000","last_undegraded":"2024-01-16T15:00:03.617703+0000","last_fullsized":"2024-01-16T15:00:03.617703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:15:58.991139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.557063+0000","last_change":"2024-01-16T15:00:03.581190+0000","last_active":"2024-01-16T15:00:04.557063+0000","last_peered":"2024-01-16T15:00:04.557063+0000","last_clean":"2024-01-16T15:00:04.557063+0000","last_became_active":"2024-01-16T15:00:03.580903+0000","last_became_peered":"2024-01-16T15:00:03.580903+0000","last_unstale":"2024-01-16T15:00:04.557063+0000","last_undegraded":"2024-01-16T15:00:04.557063+0000","last_fullsized":"2024-01-16T15:00:04.557063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:17:11.329531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.615038+0000","last_change":"2024-01-16T15:00:02.622318+0000","last_active":"2024-01-16T15:00:03.615038+0000","last_peered":"2024-01-16T15:00:03.615038+0000","last_clean":"2024-01-16T15:00:03.615038+0000","last_became_active":"2024-01-16T15:00:02.622049+0000","last_became_peered":"2024-01-16T15:00:02.622049+0000","last_unstale":"2024-01-16T15:00:03.615038+0000","last_undegraded":"2024-01-16T15:00:03.615038+0000","last_fullsized":"2024-01-16T15:00:03.615038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:11:22.855828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323468+0000","last_change":"2024-01-16T15:00:03.574905+0000","last_active":"2024-01-16T15:00:05.323468+0000","last_peered":"2024-01-16T15:00:05.323468+0000","last_clean":"2024-01-16T15:00:05.323468+0000","last_became_active":"2024-01-16T15:00:03.572077+0000","last_became_peered":"2024-01-16T15:00:03.572077+0000","last_unstale":"2024-01-16T15:00:05.323468+0000","last_undegraded":"2024-01-16T15:00:05.323468+0000","last_fullsized":"2024-01-16T15:00:05.323468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:53:49.745969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.583051+0000","last_change":"2024-01-16T15:00:02.632349+0000","last_active":"2024-01-16T15:00:03.583051+0000","last_peered":"2024-01-16T15:00:03.583051+0000","last_clean":"2024-01-16T15:00:03.583051+0000","last_became_active":"2024-01-16T15:00:02.632038+0000","last_became_peered":"2024-01-16T15:00:02.632038+0000","last_unstale":"2024-01-16T15:00:03.583051+0000","last_undegraded":"2024-01-16T15:00:03.583051+0000","last_fullsized":"2024-01-16T15:00:03.583051+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:34:38.214930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.556735+0000","last_change":"2024-01-16T15:00:03.584616+0000","last_active":"2024-01-16T15:00:04.556735+0000","last_peered":"2024-01-16T15:00:04.556735+0000","last_clean":"2024-01-16T15:00:04.556735+0000","last_became_active":"2024-01-16T15:00:03.584348+0000","last_became_peered":"2024-01-16T15:00:03.584348+0000","last_unstale":"2024-01-16T15:00:04.556735+0000","last_undegraded":"2024-01-16T15:00:04.556735+0000","last_fullsized":"2024-01-16T15:00:04.556735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:18:16.333187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.569602+0000","last_change":"2024-01-16T15:00:02.628258+0000","last_active":"2024-01-16T15:00:03.569602+0000","last_peered":"2024-01-16T15:00:03.569602+0000","last_clean":"2024-01-16T15:00:03.569602+0000","last_became_active":"2024-01-16T15:00:02.627972+0000","last_became_peered":"2024-01-16T15:00:02.627972+0000","last_unstale":"2024-01-16T15:00:03.569602+0000","last_undegraded":"2024-01-16T15:00:03.569602+0000","last_fullsized":"2024-01-16T15:00:03.569602+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:25:27.849037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.750102+0000","last_change":"2024-01-16T15:00:03.588418+0000","last_active":"2024-01-16T15:00:04.750102+0000","last_peered":"2024-01-16T15:00:04.750102+0000","last_clean":"2024-01-16T15:00:04.750102+0000","last_became_active":"2024-01-16T15:00:03.587735+0000","last_became_peered":"2024-01-16T15:00:03.587735+0000","last_unstale":"2024-01-16T15:00:04.750102+0000","last_undegraded":"2024-01-16T15:00:04.750102+0000","last_fullsized":"2024-01-16T15:00:04.750102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:03:10.226190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.617567+0000","last_change":"2024-01-16T15:00:02.616381+0000","last_active":"2024-01-16T15:00:03.617567+0000","last_peered":"2024-01-16T15:00:03.617567+0000","last_clean":"2024-01-16T15:00:03.617567+0000","last_became_active":"2024-01-16T15:00:02.615244+0000","last_became_peered":"2024-01-16T15:00:02.615244+0000","last_unstale":"2024-01-16T15:00:03.617567+0000","last_undegraded":"2024-01-16T15:00:03.617567+0000","last_fullsized":"2024-01-16T15:00:03.617567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:21:47.228684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.556253+0000","last_change":"2024-01-16T15:00:03.584478+0000","last_active":"2024-01-16T15:00:04.556253+0000","last_peered":"2024-01-16T15:00:04.556253+0000","last_clean":"2024-01-16T15:00:04.556253+0000","last_became_active":"2024-01-16T15:00:03.583891+0000","last_became_peered":"2024-01-16T15:00:03.583891+0000","last_unstale":"2024-01-16T15:00:04.556253+0000","last_undegraded":"2024-01-16T15:00:04.556253+0000","last_fullsized":"2024-01-16T15:00:04.556253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:02:58.710795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.609993+0000","last_change":"2024-01-16T15:00:02.623236+0000","last_active":"2024-01-16T15:00:03.609993+0000","last_peered":"2024-01-16T15:00:03.609993+0000","last_clean":"2024-01-16T15:00:03.609993+0000","last_became_active":"2024-01-16T15:00:02.619961+0000","last_became_peered":"2024-01-16T15:00:02.619961+0000","last_unstale":"2024-01-16T15:00:03.609993+0000","last_undegraded":"2024-01-16T15:00:03.609993+0000","last_fullsized":"2024-01-16T15:00:03.609993+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:15:08.623284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.557756+0000","last_change":"2024-01-16T15:00:03.584135+0000","last_active":"2024-01-16T15:00:04.557756+0000","last_peered":"2024-01-16T15:00:04.557756+0000","last_clean":"2024-01-16T15:00:04.557756+0000","last_became_active":"2024-01-16T15:00:03.583742+0000","last_became_peered":"2024-01-16T15:00:03.583742+0000","last_unstale":"2024-01-16T15:00:04.557756+0000","last_undegraded":"2024-01-16T15:00:04.557756+0000","last_fullsized":"2024-01-16T15:00:04.557756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:54:11.712679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.613078+0000","last_change":"2024-01-16T15:00:02.625540+0000","last_active":"2024-01-16T15:00:03.613078+0000","last_peered":"2024-01-16T15:00:03.613078+0000","last_clean":"2024-01-16T15:00:03.613078+0000","last_became_active":"2024-01-16T15:00:02.625192+0000","last_became_peered":"2024-01-16T15:00:02.625192+0000","last_unstale":"2024-01-16T15:00:03.613078+0000","last_undegraded":"2024-01-16T15:00:03.613078+0000","last_fullsized":"2024-01-16T15:00:03.613078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:48:43.721848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.558230+0000","last_change":"2024-01-16T15:00:03.591505+0000","last_active":"2024-01-16T15:00:04.558230+0000","last_peered":"2024-01-16T15:00:04.558230+0000","last_clean":"2024-01-16T15:00:04.558230+0000","last_became_active":"2024-01-16T15:00:03.591187+0000","last_became_peered":"2024-01-16T15:00:03.591187+0000","last_unstale":"2024-01-16T15:00:04.558230+0000","last_undegraded":"2024-01-16T15:00:04.558230+0000","last_fullsized":"2024-01-16T15:00:04.558230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:05:33.003372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.610948+0000","last_change":"2024-01-16T15:00:02.627127+0000","last_active":"2024-01-16T15:00:03.610948+0000","last_peered":"2024-01-16T15:00:03.610948+0000","last_clean":"2024-01-16T15:00:03.610948+0000","last_became_active":"2024-01-16T15:00:02.624007+0000","last_became_peered":"2024-01-16T15:00:02.624007+0000","last_unstale":"2024-01-16T15:00:03.610948+0000","last_undegraded":"2024-01-16T15:00:03.610948+0000","last_fullsized":"2024-01-16T15:00:03.610948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:18:10.171130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323506+0000","last_change":"2024-01-16T15:00:03.588356+0000","last_active":"2024-01-16T15:00:05.323506+0000","last_peered":"2024-01-16T15:00:05.323506+0000","last_clean":"2024-01-16T15:00:05.323506+0000","last_became_active":"2024-01-16T15:00:03.588094+0000","last_became_peered":"2024-01-16T15:00:03.588094+0000","last_unstale":"2024-01-16T15:00:05.323506+0000","last_undegraded":"2024-01-16T15:00:05.323506+0000","last_fullsized":"2024-01-16T15:00:05.323506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:02:58.710795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.571707+0000","last_change":"2024-01-16T15:00:02.628534+0000","last_active":"2024-01-16T15:00:03.571707+0000","last_peered":"2024-01-16T15:00:03.571707+0000","last_clean":"2024-01-16T15:00:03.571707+0000","last_became_active":"2024-01-16T15:00:02.627252+0000","last_became_peered":"2024-01-16T15:00:02.627252+0000","last_unstale":"2024-01-16T15:00:03.571707+0000","last_undegraded":"2024-01-16T15:00:03.571707+0000","last_fullsized":"2024-01-16T15:00:03.571707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:17:47.960980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.556971+0000","last_change":"2024-01-16T15:00:03.588973+0000","last_active":"2024-01-16T15:00:04.556971+0000","last_peered":"2024-01-16T15:00:04.556971+0000","last_clean":"2024-01-16T15:00:04.556971+0000","last_became_active":"2024-01-16T15:00:03.588673+0000","last_became_peered":"2024-01-16T15:00:03.588673+0000","last_unstale":"2024-01-16T15:00:04.556971+0000","last_undegraded":"2024-01-16T15:00:04.556971+0000","last_fullsized":"2024-01-16T15:00:04.556971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:29:26.138536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.554975+0000","last_change":"2024-01-16T14:59:53.430715+0000","last_active":"2024-01-16T15:00:03.554975+0000","last_peered":"2024-01-16T15:00:03.554975+0000","last_clean":"2024-01-16T15:00:03.554975+0000","last_became_active":"2024-01-16T14:59:53.428209+0000","last_became_peered":"2024-01-16T14:59:53.428209+0000","last_unstale":"2024-01-16T15:00:03.554975+0000","last_undegraded":"2024-01-16T15:00:03.554975+0000","last_fullsized":"2024-01-16T15:00:03.554975+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:59:52.011232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:59:52.011232+0000","last_clean_scrub_stamp":"2024-01-16T14:59:52.011232+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:57:42.594848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.585945+0000","last_change":"2024-01-16T15:00:02.635313+0000","last_active":"2024-01-16T15:00:03.585945+0000","last_peered":"2024-01-16T15:00:03.585945+0000","last_clean":"2024-01-16T15:00:03.585945+0000","last_became_active":"2024-01-16T15:00:02.634804+0000","last_became_peered":"2024-01-16T15:00:02.634804+0000","last_unstale":"2024-01-16T15:00:03.585945+0000","last_undegraded":"2024-01-16T15:00:03.585945+0000","last_fullsized":"2024-01-16T15:00:03.585945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:28:02.197595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.127292+0000","last_change":"2024-01-16T15:00:03.590575+0000","last_active":"2024-01-16T15:00:05.127292+0000","last_peered":"2024-01-16T15:00:05.127292+0000","last_clean":"2024-01-16T15:00:05.127292+0000","last_became_active":"2024-01-16T15:00:03.590218+0000","last_became_peered":"2024-01-16T15:00:03.590218+0000","last_unstale":"2024-01-16T15:00:05.127292+0000","last_undegraded":"2024-01-16T15:00:05.127292+0000","last_fullsized":"2024-01-16T15:00:05.127292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:33:59.253631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.555195+0000","last_change":"2024-01-16T15:00:02.636675+0000","last_active":"2024-01-16T15:00:03.555195+0000","last_peered":"2024-01-16T15:00:03.555195+0000","last_clean":"2024-01-16T15:00:03.555195+0000","last_became_active":"2024-01-16T15:00:02.633102+0000","last_became_peered":"2024-01-16T15:00:02.633102+0000","last_unstale":"2024-01-16T15:00:03.555195+0000","last_undegraded":"2024-01-16T15:00:03.555195+0000","last_fullsized":"2024-01-16T15:00:03.555195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:57:13.913346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323042+0000","last_change":"2024-01-16T15:00:03.583866+0000","last_active":"2024-01-16T15:00:05.323042+0000","last_peered":"2024-01-16T15:00:05.323042+0000","last_clean":"2024-01-16T15:00:05.323042+0000","last_became_active":"2024-01-16T15:00:03.583561+0000","last_became_peered":"2024-01-16T15:00:03.583561+0000","last_unstale":"2024-01-16T15:00:05.323042+0000","last_undegraded":"2024-01-16T15:00:05.323042+0000","last_fullsized":"2024-01-16T15:00:05.323042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:25:28.863720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.571916+0000","last_change":"2024-01-16T15:00:02.624133+0000","last_active":"2024-01-16T15:00:03.571916+0000","last_peered":"2024-01-16T15:00:03.571916+0000","last_clean":"2024-01-16T15:00:03.571916+0000","last_became_active":"2024-01-16T15:00:02.623883+0000","last_became_peered":"2024-01-16T15:00:02.623883+0000","last_unstale":"2024-01-16T15:00:03.571916+0000","last_undegraded":"2024-01-16T15:00:03.571916+0000","last_fullsized":"2024-01-16T15:00:03.571916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:00:52.721183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.554313+0000","last_change":"2024-01-16T15:00:03.571151+0000","last_active":"2024-01-16T15:00:04.554313+0000","last_peered":"2024-01-16T15:00:04.554313+0000","last_clean":"2024-01-16T15:00:04.554313+0000","last_became_active":"2024-01-16T15:00:03.570800+0000","last_became_peered":"2024-01-16T15:00:03.570800+0000","last_unstale":"2024-01-16T15:00:04.554313+0000","last_undegraded":"2024-01-16T15:00:04.554313+0000","last_fullsized":"2024-01-16T15:00:04.554313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:03:10.226190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.586154+0000","last_change":"2024-01-16T15:00:02.637246+0000","last_active":"2024-01-16T15:00:03.586154+0000","last_peered":"2024-01-16T15:00:03.586154+0000","last_clean":"2024-01-16T15:00:03.586154+0000","last_became_active":"2024-01-16T15:00:02.636465+0000","last_became_peered":"2024-01-16T15:00:02.636465+0000","last_unstale":"2024-01-16T15:00:03.586154+0000","last_undegraded":"2024-01-16T15:00:03.586154+0000","last_fullsized":"2024-01-16T15:00:03.586154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:33:58.238948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.555119+0000","last_change":"2024-01-16T15:00:03.580644+0000","last_active":"2024-01-16T15:00:04.555119+0000","last_peered":"2024-01-16T15:00:04.555119+0000","last_clean":"2024-01-16T15:00:04.555119+0000","last_became_active":"2024-01-16T15:00:03.573120+0000","last_became_peered":"2024-01-16T15:00:03.573120+0000","last_unstale":"2024-01-16T15:00:04.555119+0000","last_undegraded":"2024-01-16T15:00:04.555119+0000","last_fullsized":"2024-01-16T15:00:04.555119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:21:48.243367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.563287+0000","last_change":"2024-01-16T15:00:02.620390+0000","last_active":"2024-01-16T15:00:03.563287+0000","last_peered":"2024-01-16T15:00:03.563287+0000","last_clean":"2024-01-16T15:00:03.563287+0000","last_became_active":"2024-01-16T15:00:02.619046+0000","last_became_peered":"2024-01-16T15:00:02.619046+0000","last_unstale":"2024-01-16T15:00:03.563287+0000","last_undegraded":"2024-01-16T15:00:03.563287+0000","last_fullsized":"2024-01-16T15:00:03.563287+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:48:43.721848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323450+0000","last_change":"2024-01-16T15:00:03.587239+0000","last_active":"2024-01-16T15:00:05.323450+0000","last_peered":"2024-01-16T15:00:05.323450+0000","last_clean":"2024-01-16T15:00:05.323450+0000","last_became_active":"2024-01-16T15:00:03.586925+0000","last_became_peered":"2024-01-16T15:00:03.586925+0000","last_unstale":"2024-01-16T15:00:05.323450+0000","last_undegraded":"2024-01-16T15:00:05.323450+0000","last_fullsized":"2024-01-16T15:00:05.323450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:09:47.428018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.555099+0000","last_change":"2024-01-16T15:00:02.640889+0000","last_active":"2024-01-16T15:00:03.555099+0000","last_peered":"2024-01-16T15:00:03.555099+0000","last_clean":"2024-01-16T15:00:03.555099+0000","last_became_active":"2024-01-16T15:00:02.633676+0000","last_became_peered":"2024-01-16T15:00:02.633676+0000","last_unstale":"2024-01-16T15:00:03.555099+0000","last_undegraded":"2024-01-16T15:00:03.555099+0000","last_fullsized":"2024-01-16T15:00:03.555099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:03:09.211508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.554126+0000","last_change":"2024-01-16T15:00:03.576742+0000","last_active":"2024-01-16T15:00:04.554126+0000","last_peered":"2024-01-16T15:00:04.554126+0000","last_clean":"2024-01-16T15:00:04.554126+0000","last_became_active":"2024-01-16T15:00:03.576104+0000","last_became_peered":"2024-01-16T15:00:03.576104+0000","last_unstale":"2024-01-16T15:00:04.554126+0000","last_undegraded":"2024-01-16T15:00:04.554126+0000","last_fullsized":"2024-01-16T15:00:04.554126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:28:03.212278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.554530+0000","last_change":"2024-01-16T15:00:02.633317+0000","last_active":"2024-01-16T15:00:04.554530+0000","last_peered":"2024-01-16T15:00:04.554530+0000","last_clean":"2024-01-16T15:00:04.554530+0000","last_became_active":"2024-01-16T15:00:02.632996+0000","last_became_peered":"2024-01-16T15:00:02.632996+0000","last_unstale":"2024-01-16T15:00:04.554530+0000","last_undegraded":"2024-01-16T15:00:04.554530+0000","last_fullsized":"2024-01-16T15:00:04.554530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:53:48.731286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.128153+0000","last_change":"2024-01-16T15:00:03.577601+0000","last_active":"2024-01-16T15:00:05.128153+0000","last_peered":"2024-01-16T15:00:05.128153+0000","last_clean":"2024-01-16T15:00:05.128153+0000","last_became_active":"2024-01-16T15:00:03.577107+0000","last_became_peered":"2024-01-16T15:00:03.577107+0000","last_unstale":"2024-01-16T15:00:05.128153+0000","last_undegraded":"2024-01-16T15:00:05.128153+0000","last_fullsized":"2024-01-16T15:00:05.128153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:31:09.422997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.612223+0000","last_change":"2024-01-16T15:00:02.634633+0000","last_active":"2024-01-16T15:00:03.612223+0000","last_peered":"2024-01-16T15:00:03.612223+0000","last_clean":"2024-01-16T15:00:03.612223+0000","last_became_active":"2024-01-16T15:00:02.634372+0000","last_became_peered":"2024-01-16T15:00:02.634372+0000","last_unstale":"2024-01-16T15:00:03.612223+0000","last_undegraded":"2024-01-16T15:00:03.612223+0000","last_fullsized":"2024-01-16T15:00:03.612223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:11:10.313458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.324092+0000","last_change":"2024-01-16T15:00:03.575017+0000","last_active":"2024-01-16T15:00:05.324092+0000","last_peered":"2024-01-16T15:00:05.324092+0000","last_clean":"2024-01-16T15:00:05.324092+0000","last_became_active":"2024-01-16T15:00:03.573750+0000","last_became_peered":"2024-01-16T15:00:03.573750+0000","last_unstale":"2024-01-16T15:00:05.324092+0000","last_undegraded":"2024-01-16T15:00:05.324092+0000","last_fullsized":"2024-01-16T15:00:05.324092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:28:03.212278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.552344+0000","last_change":"2024-01-16T15:00:02.636963+0000","last_active":"2024-01-16T15:00:03.552344+0000","last_peered":"2024-01-16T15:00:03.552344+0000","last_clean":"2024-01-16T15:00:03.552344+0000","last_became_active":"2024-01-16T15:00:02.632840+0000","last_became_peered":"2024-01-16T15:00:02.632840+0000","last_unstale":"2024-01-16T15:00:03.552344+0000","last_undegraded":"2024-01-16T15:00:03.552344+0000","last_fullsized":"2024-01-16T15:00:03.552344+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:21:47.228684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.554498+0000","last_change":"2024-01-16T15:00:03.582851+0000","last_active":"2024-01-16T15:00:04.554498+0000","last_peered":"2024-01-16T15:00:04.554498+0000","last_clean":"2024-01-16T15:00:04.554498+0000","last_became_active":"2024-01-16T15:00:03.581868+0000","last_became_peered":"2024-01-16T15:00:03.581868+0000","last_unstale":"2024-01-16T15:00:04.554498+0000","last_undegraded":"2024-01-16T15:00:04.554498+0000","last_fullsized":"2024-01-16T15:00:04.554498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:00:53.735866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.563577+0000","last_change":"2024-01-16T15:00:02.622029+0000","last_active":"2024-01-16T15:00:03.563577+0000","last_peered":"2024-01-16T15:00:03.563577+0000","last_clean":"2024-01-16T15:00:03.563577+0000","last_became_active":"2024-01-16T15:00:02.620427+0000","last_became_peered":"2024-01-16T15:00:02.620427+0000","last_unstale":"2024-01-16T15:00:03.563577+0000","last_undegraded":"2024-01-16T15:00:03.563577+0000","last_fullsized":"2024-01-16T15:00:03.563577+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:25:27.849037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.127523+0000","last_change":"2024-01-16T15:00:03.580221+0000","last_active":"2024-01-16T15:00:05.127523+0000","last_peered":"2024-01-16T15:00:05.127523+0000","last_clean":"2024-01-16T15:00:05.127523+0000","last_became_active":"2024-01-16T15:00:03.579960+0000","last_became_peered":"2024-01-16T15:00:03.579960+0000","last_unstale":"2024-01-16T15:00:05.127523+0000","last_undegraded":"2024-01-16T15:00:05.127523+0000","last_fullsized":"2024-01-16T15:00:05.127523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:03:10.226190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.564091+0000","last_change":"2024-01-16T15:00:02.631776+0000","last_active":"2024-01-16T15:00:03.564091+0000","last_peered":"2024-01-16T15:00:03.564091+0000","last_clean":"2024-01-16T15:00:03.564091+0000","last_became_active":"2024-01-16T15:00:02.630658+0000","last_became_peered":"2024-01-16T15:00:02.630658+0000","last_unstale":"2024-01-16T15:00:03.564091+0000","last_undegraded":"2024-01-16T15:00:03.564091+0000","last_fullsized":"2024-01-16T15:00:03.564091+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:15:58.991139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.128076+0000","last_change":"2024-01-16T15:00:03.574061+0000","last_active":"2024-01-16T15:00:05.128076+0000","last_peered":"2024-01-16T15:00:05.128076+0000","last_clean":"2024-01-16T15:00:05.128076+0000","last_became_active":"2024-01-16T15:00:03.573691+0000","last_became_peered":"2024-01-16T15:00:03.573691+0000","last_unstale":"2024-01-16T15:00:05.128076+0000","last_undegraded":"2024-01-16T15:00:05.128076+0000","last_fullsized":"2024-01-16T15:00:05.128076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:05:08.127179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.615335+0000","last_change":"2024-01-16T15:00:02.634122+0000","last_active":"2024-01-16T15:00:03.615335+0000","last_peered":"2024-01-16T15:00:03.615335+0000","last_clean":"2024-01-16T15:00:03.615335+0000","last_became_active":"2024-01-16T15:00:02.633819+0000","last_became_peered":"2024-01-16T15:00:02.633819+0000","last_unstale":"2024-01-16T15:00:03.615335+0000","last_undegraded":"2024-01-16T15:00:03.615335+0000","last_fullsized":"2024-01-16T15:00:03.615335+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:53:48.731286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.750129+0000","last_change":"2024-01-16T15:00:03.590510+0000","last_active":"2024-01-16T15:00:04.750129+0000","last_peered":"2024-01-16T15:00:04.750129+0000","last_clean":"2024-01-16T15:00:04.750129+0000","last_became_active":"2024-01-16T15:00:03.588555+0000","last_became_peered":"2024-01-16T15:00:03.588555+0000","last_unstale":"2024-01-16T15:00:04.750129+0000","last_undegraded":"2024-01-16T15:00:04.750129+0000","last_fullsized":"2024-01-16T15:00:04.750129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:06:24.858403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323387+0000","last_change":"2024-01-16T15:00:03.594894+0000","last_active":"2024-01-16T15:00:05.323387+0000","last_peered":"2024-01-16T15:00:05.323387+0000","last_clean":"2024-01-16T15:00:05.323387+0000","last_became_active":"2024-01-16T15:00:03.592616+0000","last_became_peered":"2024-01-16T15:00:03.592616+0000","last_unstale":"2024-01-16T15:00:05.323387+0000","last_undegraded":"2024-01-16T15:00:05.323387+0000","last_fullsized":"2024-01-16T15:00:05.323387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:31:09.422997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.615281+0000","last_change":"2024-01-16T15:00:02.631757+0000","last_active":"2024-01-16T15:00:03.615281+0000","last_peered":"2024-01-16T15:00:03.615281+0000","last_clean":"2024-01-16T15:00:03.615281+0000","last_became_active":"2024-01-16T15:00:02.630946+0000","last_became_peered":"2024-01-16T15:00:02.630946+0000","last_unstale":"2024-01-16T15:00:03.615281+0000","last_undegraded":"2024-01-16T15:00:03.615281+0000","last_fullsized":"2024-01-16T15:00:03.615281+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:34:38.214930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323082+0000","last_change":"2024-01-16T15:00:03.593210+0000","last_active":"2024-01-16T15:00:05.323082+0000","last_peered":"2024-01-16T15:00:05.323082+0000","last_clean":"2024-01-16T15:00:05.323082+0000","last_became_active":"2024-01-16T15:00:03.592630+0000","last_became_peered":"2024-01-16T15:00:03.592630+0000","last_unstale":"2024-01-16T15:00:05.323082+0000","last_undegraded":"2024-01-16T15:00:05.323082+0000","last_fullsized":"2024-01-16T15:00:05.323082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:05:08.127179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.615363+0000","last_change":"2024-01-16T15:00:02.641351+0000","last_active":"2024-01-16T15:00:03.615363+0000","last_peered":"2024-01-16T15:00:03.615363+0000","last_clean":"2024-01-16T15:00:03.615363+0000","last_became_active":"2024-01-16T15:00:02.641021+0000","last_became_peered":"2024-01-16T15:00:02.641021+0000","last_unstale":"2024-01-16T15:00:03.615363+0000","last_undegraded":"2024-01-16T15:00:03.615363+0000","last_fullsized":"2024-01-16T15:00:03.615363+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:11:10.313458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.324164+0000","last_change":"2024-01-16T15:00:03.596490+0000","last_active":"2024-01-16T15:00:05.324164+0000","last_peered":"2024-01-16T15:00:05.324164+0000","last_clean":"2024-01-16T15:00:05.324164+0000","last_became_active":"2024-01-16T15:00:03.596137+0000","last_became_peered":"2024-01-16T15:00:03.596137+0000","last_unstale":"2024-01-16T15:00:05.324164+0000","last_undegraded":"2024-01-16T15:00:05.324164+0000","last_fullsized":"2024-01-16T15:00:05.324164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:33:59.253631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.612476+0000","last_change":"2024-01-16T15:00:02.630103+0000","last_active":"2024-01-16T15:00:03.612476+0000","last_peered":"2024-01-16T15:00:03.612476+0000","last_clean":"2024-01-16T15:00:03.612476+0000","last_became_active":"2024-01-16T15:00:02.629827+0000","last_became_peered":"2024-01-16T15:00:02.629827+0000","last_unstale":"2024-01-16T15:00:03.612476+0000","last_undegraded":"2024-01-16T15:00:03.612476+0000","last_fullsized":"2024-01-16T15:00:03.612476+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:25:27.849037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.127755+0000","last_change":"2024-01-16T15:00:03.583878+0000","last_active":"2024-01-16T15:00:05.127755+0000","last_peered":"2024-01-16T15:00:05.127755+0000","last_clean":"2024-01-16T15:00:05.127755+0000","last_became_active":"2024-01-16T15:00:03.583033+0000","last_became_peered":"2024-01-16T15:00:03.583033+0000","last_unstale":"2024-01-16T15:00:05.127755+0000","last_undegraded":"2024-01-16T15:00:05.127755+0000","last_fullsized":"2024-01-16T15:00:05.127755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:57:14.928029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.616725+0000","last_change":"2024-01-16T15:00:02.634166+0000","last_active":"2024-01-16T15:00:03.616725+0000","last_peered":"2024-01-16T15:00:03.616725+0000","last_clean":"2024-01-16T15:00:03.616725+0000","last_became_active":"2024-01-16T15:00:02.633874+0000","last_became_peered":"2024-01-16T15:00:02.633874+0000","last_unstale":"2024-01-16T15:00:03.616725+0000","last_undegraded":"2024-01-16T15:00:03.616725+0000","last_fullsized":"2024-01-16T15:00:03.616725+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:39:13.413754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1719999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2809999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.419}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.796}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3740000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8640000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7010000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.75}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1679999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2370000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.863}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.476}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.516}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0049999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6619999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.615}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.327}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0449999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.179}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3380000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.873}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5489999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2889999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1359999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3740000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5640000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95299999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1910000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":806912,"data_stored":660897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.135}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2949999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2709999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.323}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-16T15:00:13.444 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-16T15:00:13.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.702+0000 7fc936a00640 1 Processor -- start 2024-01-16T15:00:13.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.702+0000 7fc936a00640 1 -- start start 2024-01-16T15:00:13.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.704+0000 7fc936a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc930143870 0x7fc930146970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:13.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.704+0000 7fc936a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc930161080 0x7fc9300b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:13.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.704+0000 7fc936a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc930161510 con 0x7fc930146eb0 2024-01-16T15:00:13.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.704+0000 7fc936a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc930070460 con 0x7fc930143870 2024-01-16T15:00:13.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.704+0000 7fc936a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9301617f0 con 0x7fc930161080 2024-01-16T15:00:13.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.704+0000 7fc936000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc930146eb0 0x7fc93006c5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54218/0 (socket says 172.21.15.143:54218) 2024-01-16T15:00:13.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.704+0000 7fc936000640 1 -- 172.21.15.143:0/4237490687 learned_addr learned my addr 172.21.15.143:0/4237490687 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:13.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.705+0000 7fc92fe00640 1 -- 172.21.15.143:0/4237490687 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1278838445 0 0) 0x7fc930161510 con 0x7fc930146eb0 2024-01-16T15:00:13.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.705+0000 7fc934c00640 1 --2- 172.21.15.143:0/4237490687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc930161080 0x7fc9300b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:13.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.705+0000 7fc92fe00640 1 -- 172.21.15.143:0/4237490687 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc910003520 con 0x7fc930146eb0 2024-01-16T15:00:13.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.705+0000 7fc935600640 1 --2- 172.21.15.143:0/4237490687 >> v2:172.21.15.204:3300/0 conn(0x7fc930143870 0x7fc930146970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:13.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.705+0000 7fc92fe00640 1 -- 172.21.15.143:0/4237490687 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2779756889 0 0) 0x7fc910003520 con 0x7fc930146eb0 2024-01-16T15:00:13.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.705+0000 7fc92fe00640 1 -- 172.21.15.143:0/4237490687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc930161080 msgr2=0x7fc9300b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:13.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.705+0000 7fc92fe00640 1 --2- 172.21.15.143:0/4237490687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc930161080 0x7fc9300b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:13.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.705+0000 7fc92fe00640 1 -- 172.21.15.143:0/4237490687 >> v2:172.21.15.204:3300/0 conn(0x7fc930143870 msgr2=0x7fc930146970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:13.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.705+0000 7fc92fe00640 1 --2- 172.21.15.143:0/4237490687 >> v2:172.21.15.204:3300/0 conn(0x7fc930143870 0x7fc930146970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:13.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.705+0000 7fc92fe00640 1 -- 172.21.15.143:0/4237490687 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc930161ad0 con 0x7fc930146eb0 2024-01-16T15:00:13.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.705+0000 7fc92fe00640 1 -- 172.21.15.143:0/4237490687 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7fc9280019f0 con 0x7fc930146eb0 2024-01-16T15:00:13.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.705+0000 7fc92fe00640 1 -- 172.21.15.143:0/4237490687 >> v1:172.21.15.143:6789/0 conn(0x7fc930146eb0 legacy=0x7fc93006c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:13.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.705+0000 7fc92fe00640 1 --2- 172.21.15.143:0/4237490687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc910003970 0x7fc910003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:13.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.705+0000 7fc92fe00640 1 --2- 172.21.15.143:0/4237490687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9100047c0 0x7fc910004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:13.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.705+0000 7fc92fe00640 1 --2- 172.21.15.143:0/4237490687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc910005180 0x7fc91000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:13.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.705+0000 7fc92fe00640 1 -- 172.21.15.143:0/4237490687 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc930070460 con 0x7fc9100047c0 2024-01-16T15:00:13.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.705+0000 7fc92fe00640 1 -- 172.21.15.143:0/4237490687 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9301617f0 con 0x7fc910003970 2024-01-16T15:00:13.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.705+0000 7fc92fe00640 1 -- 172.21.15.143:0/4237490687 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc91000bff0 con 0x7fc910005180 2024-01-16T15:00:13.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.705+0000 7fc935600640 1 --2- 172.21.15.143:0/4237490687 >> v2:172.21.15.204:3300/0 conn(0x7fc930143870 0x7fc930146970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T15:00:13.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.705+0000 7fc934c00640 1 --2- 172.21.15.143:0/4237490687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc930161080 0x7fc9300b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T15:00:13.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.705+0000 7fc934c00640 1 --2- 172.21.15.143:0/4237490687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9100047c0 0x7fc910004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:13.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.706+0000 7fc936000640 1 --2- 172.21.15.143:0/4237490687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc910005180 0x7fc91000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:13.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.706+0000 7fc935600640 1 --2- 172.21.15.143:0/4237490687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc910003970 0x7fc910003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:13.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.706+0000 7fc934c00640 1 -- 172.21.15.143:0/4237490687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc910005180 msgr2=0x7fc91000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:13.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.706+0000 7fc934c00640 1 --2- 172.21.15.143:0/4237490687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc910005180 0x7fc91000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:13.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.706+0000 7fc934c00640 1 -- 172.21.15.143:0/4237490687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc910003970 msgr2=0x7fc910003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:13.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.706+0000 7fc934c00640 1 --2- 172.21.15.143:0/4237490687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc910003970 0x7fc910003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:13.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.706+0000 7fc934c00640 1 -- 172.21.15.143:0/4237490687 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc910003520 con 0x7fc9100047c0 2024-01-16T15:00:13.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.706+0000 7fc935600640 1 --2- 172.21.15.143:0/4237490687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc910003970 0x7fc910003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:13.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.707+0000 7fc934c00640 1 --2- 172.21.15.143:0/4237490687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9100047c0 0x7fc910004c40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc918006fd0 tx=0x7fc9100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2418d8992630a48 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:13.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.707+0000 7fc92fe00640 1 -- 172.21.15.143:0/4237490687 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc918033600 con 0x7fc9100047c0 2024-01-16T15:00:13.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.707+0000 7fc92fe00640 1 -- 172.21.15.143:0/4237490687 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc91802f050 con 0x7fc9100047c0 2024-01-16T15:00:13.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.707+0000 7fc92fe00640 1 -- 172.21.15.143:0/4237490687 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc918033980 con 0x7fc9100047c0 2024-01-16T15:00:13.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.707+0000 7fc936a00640 1 -- 172.21.15.143:0/4237490687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9100047c0 msgr2=0x7fc910004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:13.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.707+0000 7fc936a00640 1 --2- 172.21.15.143:0/4237490687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9100047c0 0x7fc910004c40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc918006fd0 tx=0x7fc9100057f0 comp rx=0 tx=0).stop 2024-01-16T15:00:13.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.707+0000 7fc936000640 1 -- 172.21.15.143:0/4237490687 reap_dead start 2024-01-16T15:00:13.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.708+0000 7fc936a00640 1 -- 172.21.15.143:0/4237490687 shutdown_connections 2024-01-16T15:00:13.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.708+0000 7fc936a00640 1 -- 172.21.15.143:0/4237490687 >> 172.21.15.143:0/4237490687 conn(0x7fc9300cb830 msgr2=0x7fc9300cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:13.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.708+0000 7fc936a00640 1 -- 172.21.15.143:0/4237490687 shutdown_connections 2024-01-16T15:00:13.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.708+0000 7fc936a00640 1 -- 172.21.15.143:0/4237490687 wait complete. 2024-01-16T15:00:13.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.709+0000 7fc936a00640 1 Processor -- start 2024-01-16T15:00:13.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.709+0000 7fc936a00640 1 -- start start 2024-01-16T15:00:13.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.710+0000 7fc936a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9100047c0 0x7fc930071860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:13.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.710+0000 7fc936a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc93015fdf0 0x7fc9300b3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:13.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.710+0000 7fc936a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc9300b38b0 0x7fc93013e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:13.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.710+0000 7fc936a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc930160a70 con 0x7fc93015fdf0 2024-01-16T15:00:13.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.710+0000 7fc936a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc930160790 con 0x7fc9100047c0 2024-01-16T15:00:13.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.710+0000 7fc936a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc93015dce0 con 0x7fc9300b38b0 2024-01-16T15:00:13.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.710+0000 7fc934c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc93015fdf0 0x7fc9300b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:13.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.710+0000 7fc934c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc93015fdf0 0x7fc9300b3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42902/0 (socket says 172.21.15.143:42902) 2024-01-16T15:00:13.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.710+0000 7fc934c00640 1 -- 172.21.15.143:0/3397902670 learned_addr learned my addr 172.21.15.143:0/3397902670 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:13.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.711+0000 7fc935600640 1 --2- 172.21.15.143:0/3397902670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9100047c0 0x7fc930071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:13.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.711+0000 7fc936000640 1 --2- 172.21.15.143:0/3397902670 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc9300b38b0 0x7fc93013e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:13.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.711+0000 7fc934c00640 1 -- 172.21.15.143:0/3397902670 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc9300b38b0 msgr2=0x7fc93013e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:13.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.711+0000 7fc934c00640 1 --2- 172.21.15.143:0/3397902670 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc9300b38b0 0x7fc93013e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:13.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.711+0000 7fc934c00640 1 -- 172.21.15.143:0/3397902670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9100047c0 msgr2=0x7fc930071860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:13.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.711+0000 7fc934c00640 1 --2- 172.21.15.143:0/3397902670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9100047c0 0x7fc930071860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:13.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.711+0000 7fc934c00640 1 -- 172.21.15.143:0/3397902670 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc93015dfc0 con 0x7fc93015fdf0 2024-01-16T15:00:13.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.711+0000 7fc935600640 1 --2- 172.21.15.143:0/3397902670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9100047c0 0x7fc930071860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:13.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.711+0000 7fc934c00640 1 --2- 172.21.15.143:0/3397902670 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc93015fdf0 0x7fc9300b3370 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc93006dd50 tx=0x7fc918004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:13.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.711+0000 7fc92d600640 1 -- 172.21.15.143:0/3397902670 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91803e830 con 0x7fc93015fdf0 2024-01-16T15:00:13.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.711+0000 7fc92d600640 1 -- 172.21.15.143:0/3397902670 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc91803e990 con 0x7fc93015fdf0 2024-01-16T15:00:13.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.711+0000 7fc92d600640 1 -- 172.21.15.143:0/3397902670 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91803ed40 con 0x7fc93015fdf0 2024-01-16T15:00:13.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.711+0000 7fc936a00640 1 -- 172.21.15.143:0/3397902670 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc93015e2a0 con 0x7fc93015fdf0 2024-01-16T15:00:13.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.712+0000 7fc936a00640 1 -- 172.21.15.143:0/3397902670 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc930071f80 con 0x7fc93015fdf0 2024-01-16T15:00:13.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.713+0000 7fc936a00640 1 -- 172.21.15.143:0/3397902670 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc93015db40 con 0x7fc93015fdf0 2024-01-16T15:00:13.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.713+0000 7fc92d600640 1 -- 172.21.15.143:0/3397902670 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fc918047460 con 0x7fc93015fdf0 2024-01-16T15:00:13.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.714+0000 7fc92d600640 1 --2- 172.21.15.143:0/3397902670 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fc90806b2c0 0x7fc90806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:13.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.714+0000 7fc92d600640 1 -- 172.21.15.143:0/3397902670 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc9180ac070 con 0x7fc93015fdf0 2024-01-16T15:00:13.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.714+0000 7fc935600640 1 --2- 172.21.15.143:0/3397902670 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fc90806b2c0 0x7fc90806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:13.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.715+0000 7fc935600640 1 --2- 172.21.15.143:0/3397902670 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fc90806b2c0 0x7fc90806d780 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc920006e80 tx=0x7fc920008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:13.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.719+0000 7fc92d600640 1 -- 172.21.15.143:0/3397902670 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc93015db40 con 0x7fc93015fdf0 2024-01-16T15:00:13.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.891+0000 7fc936a00640 1 -- 172.21.15.143:0/3397902670 --> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc930143c50 con 0x7fc90806b2c0 2024-01-16T15:00:13.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.899+0000 7fc92d600640 1 -- 172.21.15.143:0/3397902670 <== mgr.4113 v2:172.21.15.143:6832/3153492115 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264330 (secure 0 0 0) 0x7fc92000c040 con 0x7fc90806b2c0 2024-01-16T15:00:13.884 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:00:13.885 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-16T15:00:13.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.904+0000 7fc936a00640 1 -- 172.21.15.143:0/3397902670 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fc90806b2c0 msgr2=0x7fc90806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:13.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.904+0000 7fc936a00640 1 --2- 172.21.15.143:0/3397902670 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fc90806b2c0 0x7fc90806d780 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc920006e80 tx=0x7fc920008040 comp rx=0 tx=0).stop 2024-01-16T15:00:13.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.904+0000 7fc936a00640 1 -- 172.21.15.143:0/3397902670 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc93015fdf0 msgr2=0x7fc9300b3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:13.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.904+0000 7fc936a00640 1 --2- 172.21.15.143:0/3397902670 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc93015fdf0 0x7fc9300b3370 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc93006dd50 tx=0x7fc918004730 comp rx=0 tx=0).stop 2024-01-16T15:00:13.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.904+0000 7fc936a00640 1 -- 172.21.15.143:0/3397902670 shutdown_connections 2024-01-16T15:00:13.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.904+0000 7fc936a00640 1 --2- 172.21.15.143:0/3397902670 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fc90806b2c0 0x7fc90806d780 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:13.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.904+0000 7fc936a00640 1 --2- 172.21.15.143:0/3397902670 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc9300b38b0 0x7fc93013e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:13.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.904+0000 7fc936a00640 1 --2- 172.21.15.143:0/3397902670 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc93015fdf0 0x7fc9300b3370 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:13.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.904+0000 7fc936a00640 1 --2- 172.21.15.143:0/3397902670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9100047c0 0x7fc930071860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:13.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.904+0000 7fc936a00640 1 -- 172.21.15.143:0/3397902670 >> 172.21.15.143:0/3397902670 conn(0x7fc9300cb830 msgr2=0x7fc930143e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:13.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.904+0000 7fc936a00640 1 -- 172.21.15.143:0/3397902670 shutdown_connections 2024-01-16T15:00:13.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:13.904+0000 7fc936a00640 1 -- 172.21.15.143:0/3397902670 wait complete. 2024-01-16T15:00:13.902 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-01-16T15:00:13.655866+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219004,"kb_used_data":2820,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755716,"statfs":{"total":773094113280,"available":772869853184,"internally_reserved":0,"allocated":2887680,"data_stored":1751817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002737"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.558033+0000","last_change":"2024-01-16T15:00:03.579227+0000","last_active":"2024-01-16T15:00:04.558033+0000","last_peered":"2024-01-16T15:00:04.558033+0000","last_clean":"2024-01-16T15:00:04.558033+0000","last_became_active":"2024-01-16T15:00:03.578921+0000","last_became_peered":"2024-01-16T15:00:03.578921+0000","last_unstale":"2024-01-16T15:00:04.558033+0000","last_undegraded":"2024-01-16T15:00:04.558033+0000","last_fullsized":"2024-01-16T15:00:04.558033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:32:18.387639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.554616+0000","last_change":"2024-01-16T15:00:03.571130+0000","last_active":"2024-01-16T15:00:04.554616+0000","last_peered":"2024-01-16T15:00:04.554616+0000","last_clean":"2024-01-16T15:00:04.554616+0000","last_became_active":"2024-01-16T15:00:03.570816+0000","last_became_peered":"2024-01-16T15:00:03.570816+0000","last_unstale":"2024-01-16T15:00:04.554616+0000","last_undegraded":"2024-01-16T15:00:04.554616+0000","last_fullsized":"2024-01-16T15:00:04.554616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:06:24.858403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323971+0000","last_change":"2024-01-16T15:00:03.578171+0000","last_active":"2024-01-16T15:00:05.323971+0000","last_peered":"2024-01-16T15:00:05.323971+0000","last_clean":"2024-01-16T15:00:05.323971+0000","last_became_active":"2024-01-16T15:00:03.577849+0000","last_became_peered":"2024-01-16T15:00:03.577849+0000","last_unstale":"2024-01-16T15:00:05.323971+0000","last_undegraded":"2024-01-16T15:00:05.323971+0000","last_fullsized":"2024-01-16T15:00:05.323971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:11:11.328141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323719+0000","last_change":"2024-01-16T15:00:03.585019+0000","last_active":"2024-01-16T15:00:05.323719+0000","last_peered":"2024-01-16T15:00:05.323719+0000","last_clean":"2024-01-16T15:00:05.323719+0000","last_became_active":"2024-01-16T15:00:03.584747+0000","last_became_peered":"2024-01-16T15:00:03.584747+0000","last_unstale":"2024-01-16T15:00:05.323719+0000","last_undegraded":"2024-01-16T15:00:05.323719+0000","last_fullsized":"2024-01-16T15:00:05.323719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:44:37.235476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.556701+0000","last_change":"2024-01-16T15:00:03.577404+0000","last_active":"2024-01-16T15:00:04.556701+0000","last_peered":"2024-01-16T15:00:04.556701+0000","last_clean":"2024-01-16T15:00:04.556701+0000","last_became_active":"2024-01-16T15:00:03.577051+0000","last_became_peered":"2024-01-16T15:00:03.577051+0000","last_unstale":"2024-01-16T15:00:04.556701+0000","last_undegraded":"2024-01-16T15:00:04.556701+0000","last_fullsized":"2024-01-16T15:00:04.556701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:44:37.235476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.322548+0000","last_change":"2024-01-16T15:00:03.589065+0000","last_active":"2024-01-16T15:00:05.322548+0000","last_peered":"2024-01-16T15:00:05.322548+0000","last_clean":"2024-01-16T15:00:05.322548+0000","last_became_active":"2024-01-16T15:00:03.588776+0000","last_became_peered":"2024-01-16T15:00:03.588776+0000","last_unstale":"2024-01-16T15:00:05.322548+0000","last_undegraded":"2024-01-16T15:00:05.322548+0000","last_fullsized":"2024-01-16T15:00:05.322548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:06:24.858403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.556141+0000","last_change":"2024-01-16T15:00:03.570611+0000","last_active":"2024-01-16T15:00:04.556141+0000","last_peered":"2024-01-16T15:00:04.556141+0000","last_clean":"2024-01-16T15:00:04.556141+0000","last_became_active":"2024-01-16T15:00:03.569839+0000","last_became_peered":"2024-01-16T15:00:03.569839+0000","last_unstale":"2024-01-16T15:00:04.556141+0000","last_undegraded":"2024-01-16T15:00:04.556141+0000","last_fullsized":"2024-01-16T15:00:04.556141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:09:47.428018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.127040+0000","last_change":"2024-01-16T15:00:03.586954+0000","last_active":"2024-01-16T15:00:05.127040+0000","last_peered":"2024-01-16T15:00:05.127040+0000","last_clean":"2024-01-16T15:00:05.127040+0000","last_became_active":"2024-01-16T15:00:03.586663+0000","last_became_peered":"2024-01-16T15:00:03.586663+0000","last_unstale":"2024-01-16T15:00:05.127040+0000","last_undegraded":"2024-01-16T15:00:05.127040+0000","last_fullsized":"2024-01-16T15:00:05.127040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:34:39.229613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.558126+0000","last_change":"2024-01-16T15:00:03.596120+0000","last_active":"2024-01-16T15:00:04.558126+0000","last_peered":"2024-01-16T15:00:04.558126+0000","last_clean":"2024-01-16T15:00:04.558126+0000","last_became_active":"2024-01-16T15:00:03.595866+0000","last_became_peered":"2024-01-16T15:00:03.595866+0000","last_unstale":"2024-01-16T15:00:04.558126+0000","last_undegraded":"2024-01-16T15:00:04.558126+0000","last_fullsized":"2024-01-16T15:00:04.558126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:53:57.952288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.750726+0000","last_change":"2024-01-16T15:00:03.588019+0000","last_active":"2024-01-16T15:00:04.750726+0000","last_peered":"2024-01-16T15:00:04.750726+0000","last_clean":"2024-01-16T15:00:04.750726+0000","last_became_active":"2024-01-16T15:00:03.587359+0000","last_became_peered":"2024-01-16T15:00:03.587359+0000","last_unstale":"2024-01-16T15:00:04.750726+0000","last_undegraded":"2024-01-16T15:00:04.750726+0000","last_fullsized":"2024-01-16T15:00:04.750726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:31:56.335251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.750616+0000","last_change":"2024-01-16T15:00:03.590950+0000","last_active":"2024-01-16T15:00:04.750616+0000","last_peered":"2024-01-16T15:00:04.750616+0000","last_clean":"2024-01-16T15:00:04.750616+0000","last_became_active":"2024-01-16T15:00:03.590702+0000","last_became_peered":"2024-01-16T15:00:03.590702+0000","last_unstale":"2024-01-16T15:00:04.750616+0000","last_undegraded":"2024-01-16T15:00:04.750616+0000","last_fullsized":"2024-01-16T15:00:04.750616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:04:13.308774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.555500+0000","last_change":"2024-01-16T15:00:03.582929+0000","last_active":"2024-01-16T15:00:04.555500+0000","last_peered":"2024-01-16T15:00:04.555500+0000","last_clean":"2024-01-16T15:00:04.555500+0000","last_became_active":"2024-01-16T15:00:03.582040+0000","last_became_peered":"2024-01-16T15:00:03.582040+0000","last_unstale":"2024-01-16T15:00:04.555500+0000","last_undegraded":"2024-01-16T15:00:04.555500+0000","last_fullsized":"2024-01-16T15:00:04.555500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:10:55.291745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.127726+0000","last_change":"2024-01-16T15:00:03.581183+0000","last_active":"2024-01-16T15:00:05.127726+0000","last_peered":"2024-01-16T15:00:05.127726+0000","last_clean":"2024-01-16T15:00:05.127726+0000","last_became_active":"2024-01-16T15:00:03.578642+0000","last_became_peered":"2024-01-16T15:00:03.578642+0000","last_unstale":"2024-01-16T15:00:05.127726+0000","last_undegraded":"2024-01-16T15:00:05.127726+0000","last_fullsized":"2024-01-16T15:00:05.127726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:06:24.858403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323409+0000","last_change":"2024-01-16T15:00:03.570282+0000","last_active":"2024-01-16T15:00:05.323409+0000","last_peered":"2024-01-16T15:00:05.323409+0000","last_clean":"2024-01-16T15:00:05.323409+0000","last_became_active":"2024-01-16T15:00:03.569988+0000","last_became_peered":"2024-01-16T15:00:03.569988+0000","last_unstale":"2024-01-16T15:00:05.323409+0000","last_undegraded":"2024-01-16T15:00:05.323409+0000","last_fullsized":"2024-01-16T15:00:05.323409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:16:00.005822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.554056+0000","last_change":"2024-01-16T15:00:03.584705+0000","last_active":"2024-01-16T15:00:04.554056+0000","last_peered":"2024-01-16T15:00:04.554056+0000","last_clean":"2024-01-16T15:00:04.554056+0000","last_became_active":"2024-01-16T15:00:03.584361+0000","last_became_peered":"2024-01-16T15:00:03.584361+0000","last_unstale":"2024-01-16T15:00:04.554056+0000","last_undegraded":"2024-01-16T15:00:04.554056+0000","last_fullsized":"2024-01-16T15:00:04.554056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:11:11.328141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.322398+0000","last_change":"2024-01-16T15:00:03.587177+0000","last_active":"2024-01-16T15:00:05.322398+0000","last_peered":"2024-01-16T15:00:05.322398+0000","last_clean":"2024-01-16T15:00:05.322398+0000","last_became_active":"2024-01-16T15:00:03.586803+0000","last_became_peered":"2024-01-16T15:00:03.586803+0000","last_unstale":"2024-01-16T15:00:05.322398+0000","last_undegraded":"2024-01-16T15:00:05.322398+0000","last_fullsized":"2024-01-16T15:00:05.322398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:03:10.226190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323936+0000","last_change":"2024-01-16T15:00:03.595310+0000","last_active":"2024-01-16T15:00:05.323936+0000","last_peered":"2024-01-16T15:00:05.323936+0000","last_clean":"2024-01-16T15:00:05.323936+0000","last_became_active":"2024-01-16T15:00:03.589586+0000","last_became_peered":"2024-01-16T15:00:03.589586+0000","last_unstale":"2024-01-16T15:00:05.323936+0000","last_undegraded":"2024-01-16T15:00:05.323936+0000","last_fullsized":"2024-01-16T15:00:05.323936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:29:26.138536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.322465+0000","last_change":"2024-01-16T15:00:03.583181+0000","last_active":"2024-01-16T15:00:05.322465+0000","last_peered":"2024-01-16T15:00:05.322465+0000","last_clean":"2024-01-16T15:00:05.322465+0000","last_became_active":"2024-01-16T15:00:03.582404+0000","last_became_peered":"2024-01-16T15:00:03.582404+0000","last_unstale":"2024-01-16T15:00:05.322465+0000","last_undegraded":"2024-01-16T15:00:05.322465+0000","last_fullsized":"2024-01-16T15:00:05.322465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:25:28.863720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.557296+0000","last_change":"2024-01-16T15:00:03.577376+0000","last_active":"2024-01-16T15:00:04.557296+0000","last_peered":"2024-01-16T15:00:04.557296+0000","last_clean":"2024-01-16T15:00:04.557296+0000","last_became_active":"2024-01-16T15:00:03.577167+0000","last_became_peered":"2024-01-16T15:00:03.577167+0000","last_unstale":"2024-01-16T15:00:04.557296+0000","last_undegraded":"2024-01-16T15:00:04.557296+0000","last_fullsized":"2024-01-16T15:00:04.557296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:14:46.103689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.750577+0000","last_change":"2024-01-16T15:00:03.587122+0000","last_active":"2024-01-16T15:00:04.750577+0000","last_peered":"2024-01-16T15:00:04.750577+0000","last_clean":"2024-01-16T15:00:04.750577+0000","last_became_active":"2024-01-16T15:00:03.586244+0000","last_became_peered":"2024-01-16T15:00:03.586244+0000","last_unstale":"2024-01-16T15:00:04.750577+0000","last_undegraded":"2024-01-16T15:00:04.750577+0000","last_fullsized":"2024-01-16T15:00:04.750577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:57:14.928029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323320+0000","last_change":"2024-01-16T15:00:03.572186+0000","last_active":"2024-01-16T15:00:05.323320+0000","last_peered":"2024-01-16T15:00:05.323320+0000","last_clean":"2024-01-16T15:00:05.323320+0000","last_became_active":"2024-01-16T15:00:03.571795+0000","last_became_peered":"2024-01-16T15:00:03.571795+0000","last_unstale":"2024-01-16T15:00:05.323320+0000","last_undegraded":"2024-01-16T15:00:05.323320+0000","last_fullsized":"2024-01-16T15:00:05.323320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:18:11.185813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.322924+0000","last_change":"2024-01-16T15:00:03.589661+0000","last_active":"2024-01-16T15:00:05.322924+0000","last_peered":"2024-01-16T15:00:05.322924+0000","last_clean":"2024-01-16T15:00:05.322924+0000","last_became_active":"2024-01-16T15:00:03.589365+0000","last_became_peered":"2024-01-16T15:00:03.589365+0000","last_unstale":"2024-01-16T15:00:05.322924+0000","last_undegraded":"2024-01-16T15:00:05.322924+0000","last_fullsized":"2024-01-16T15:00:05.322924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:04:13.308774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.127047+0000","last_change":"2024-01-16T15:00:03.591522+0000","last_active":"2024-01-16T15:00:05.127047+0000","last_peered":"2024-01-16T15:00:05.127047+0000","last_clean":"2024-01-16T15:00:05.127047+0000","last_became_active":"2024-01-16T15:00:03.591143+0000","last_became_peered":"2024-01-16T15:00:03.591143+0000","last_unstale":"2024-01-16T15:00:05.127047+0000","last_undegraded":"2024-01-16T15:00:05.127047+0000","last_fullsized":"2024-01-16T15:00:05.127047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:11:11.328141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.322833+0000","last_change":"2024-01-16T15:00:03.585381+0000","last_active":"2024-01-16T15:00:05.322833+0000","last_peered":"2024-01-16T15:00:05.322833+0000","last_clean":"2024-01-16T15:00:05.322833+0000","last_became_active":"2024-01-16T15:00:03.584858+0000","last_became_peered":"2024-01-16T15:00:03.584858+0000","last_unstale":"2024-01-16T15:00:05.322833+0000","last_undegraded":"2024-01-16T15:00:05.322833+0000","last_fullsized":"2024-01-16T15:00:05.322833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:57:14.928029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.127981+0000","last_change":"2024-01-16T15:00:03.591627+0000","last_active":"2024-01-16T15:00:05.127981+0000","last_peered":"2024-01-16T15:00:05.127981+0000","last_clean":"2024-01-16T15:00:05.127981+0000","last_became_active":"2024-01-16T15:00:03.591216+0000","last_became_peered":"2024-01-16T15:00:03.591216+0000","last_unstale":"2024-01-16T15:00:05.127981+0000","last_undegraded":"2024-01-16T15:00:05.127981+0000","last_fullsized":"2024-01-16T15:00:05.127981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:31:56.335251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.750822+0000","last_change":"2024-01-16T15:00:03.588262+0000","last_active":"2024-01-16T15:00:04.750822+0000","last_peered":"2024-01-16T15:00:04.750822+0000","last_clean":"2024-01-16T15:00:04.750822+0000","last_became_active":"2024-01-16T15:00:03.587728+0000","last_became_peered":"2024-01-16T15:00:03.587728+0000","last_unstale":"2024-01-16T15:00:04.750822+0000","last_undegraded":"2024-01-16T15:00:04.750822+0000","last_fullsized":"2024-01-16T15:00:04.750822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:05:08.127179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.554267+0000","last_change":"2024-01-16T15:00:03.576892+0000","last_active":"2024-01-16T15:00:04.554267+0000","last_peered":"2024-01-16T15:00:04.554267+0000","last_clean":"2024-01-16T15:00:04.554267+0000","last_became_active":"2024-01-16T15:00:03.574566+0000","last_became_peered":"2024-01-16T15:00:03.574566+0000","last_unstale":"2024-01-16T15:00:04.554267+0000","last_undegraded":"2024-01-16T15:00:04.554267+0000","last_fullsized":"2024-01-16T15:00:04.554267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:33:59.253631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.127898+0000","last_change":"2024-01-16T15:00:03.591601+0000","last_active":"2024-01-16T15:00:05.127898+0000","last_peered":"2024-01-16T15:00:05.127898+0000","last_clean":"2024-01-16T15:00:05.127898+0000","last_became_active":"2024-01-16T15:00:03.591271+0000","last_became_peered":"2024-01-16T15:00:03.591271+0000","last_unstale":"2024-01-16T15:00:05.127898+0000","last_undegraded":"2024-01-16T15:00:05.127898+0000","last_fullsized":"2024-01-16T15:00:05.127898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:04:13.308774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323503+0000","last_change":"2024-01-16T15:00:03.580655+0000","last_active":"2024-01-16T15:00:05.323503+0000","last_peered":"2024-01-16T15:00:05.323503+0000","last_clean":"2024-01-16T15:00:05.323503+0000","last_became_active":"2024-01-16T15:00:03.580091+0000","last_became_peered":"2024-01-16T15:00:03.580091+0000","last_unstale":"2024-01-16T15:00:05.323503+0000","last_undegraded":"2024-01-16T15:00:05.323503+0000","last_fullsized":"2024-01-16T15:00:05.323503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:34:39.229613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.322959+0000","last_change":"2024-01-16T15:00:03.588486+0000","last_active":"2024-01-16T15:00:05.322959+0000","last_peered":"2024-01-16T15:00:05.322959+0000","last_clean":"2024-01-16T15:00:05.322959+0000","last_became_active":"2024-01-16T15:00:03.586447+0000","last_became_peered":"2024-01-16T15:00:03.586447+0000","last_unstale":"2024-01-16T15:00:05.322959+0000","last_undegraded":"2024-01-16T15:00:05.322959+0000","last_fullsized":"2024-01-16T15:00:05.322959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:31:56.335251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.322016+0000","last_change":"2024-01-16T15:00:03.581151+0000","last_active":"2024-01-16T15:00:05.322016+0000","last_peered":"2024-01-16T15:00:05.322016+0000","last_clean":"2024-01-16T15:00:05.322016+0000","last_became_active":"2024-01-16T15:00:03.579463+0000","last_became_peered":"2024-01-16T15:00:03.579463+0000","last_unstale":"2024-01-16T15:00:05.322016+0000","last_undegraded":"2024-01-16T15:00:05.322016+0000","last_fullsized":"2024-01-16T15:00:05.322016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:10:55.291745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.750064+0000","last_change":"2024-01-16T15:00:03.588682+0000","last_active":"2024-01-16T15:00:04.750064+0000","last_peered":"2024-01-16T15:00:04.750064+0000","last_clean":"2024-01-16T15:00:04.750064+0000","last_became_active":"2024-01-16T15:00:03.588148+0000","last_became_peered":"2024-01-16T15:00:03.588148+0000","last_unstale":"2024-01-16T15:00:04.750064+0000","last_undegraded":"2024-01-16T15:00:04.750064+0000","last_fullsized":"2024-01-16T15:00:04.750064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:33:59.253631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.555160+0000","last_change":"2024-01-16T15:00:03.580412+0000","last_active":"2024-01-16T15:00:04.555160+0000","last_peered":"2024-01-16T15:00:04.555160+0000","last_clean":"2024-01-16T15:00:04.555160+0000","last_became_active":"2024-01-16T15:00:03.572852+0000","last_became_peered":"2024-01-16T15:00:03.572852+0000","last_unstale":"2024-01-16T15:00:04.555160+0000","last_undegraded":"2024-01-16T15:00:04.555160+0000","last_fullsized":"2024-01-16T15:00:04.555160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:48:44.736531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.613598+0000","last_change":"2024-01-16T15:00:02.635074+0000","last_active":"2024-01-16T15:00:03.613598+0000","last_peered":"2024-01-16T15:00:03.613598+0000","last_clean":"2024-01-16T15:00:03.613598+0000","last_became_active":"2024-01-16T15:00:02.634767+0000","last_became_peered":"2024-01-16T15:00:02.634767+0000","last_unstale":"2024-01-16T15:00:03.613598+0000","last_undegraded":"2024-01-16T15:00:03.613598+0000","last_fullsized":"2024-01-16T15:00:03.613598+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:15:58.991139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.324292+0000","last_change":"2024-01-16T15:00:03.573187+0000","last_active":"2024-01-16T15:00:05.324292+0000","last_peered":"2024-01-16T15:00:05.324292+0000","last_clean":"2024-01-16T15:00:05.324292+0000","last_became_active":"2024-01-16T15:00:03.572755+0000","last_became_peered":"2024-01-16T15:00:03.572755+0000","last_unstale":"2024-01-16T15:00:05.324292+0000","last_undegraded":"2024-01-16T15:00:05.324292+0000","last_fullsized":"2024-01-16T15:00:05.324292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:06:24.858403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.610883+0000","last_change":"2024-01-16T15:00:02.632028+0000","last_active":"2024-01-16T15:00:03.610883+0000","last_peered":"2024-01-16T15:00:03.610883+0000","last_clean":"2024-01-16T15:00:03.610883+0000","last_became_active":"2024-01-16T15:00:02.626019+0000","last_became_peered":"2024-01-16T15:00:02.626019+0000","last_unstale":"2024-01-16T15:00:03.610883+0000","last_undegraded":"2024-01-16T15:00:03.610883+0000","last_fullsized":"2024-01-16T15:00:03.610883+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:48:43.721848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.568172+0000","last_change":"2024-01-16T15:00:02.635363+0000","last_active":"2024-01-16T15:00:03.568172+0000","last_peered":"2024-01-16T15:00:03.568172+0000","last_clean":"2024-01-16T15:00:03.568172+0000","last_became_active":"2024-01-16T15:00:02.634387+0000","last_became_peered":"2024-01-16T15:00:02.634387+0000","last_unstale":"2024-01-16T15:00:03.568172+0000","last_undegraded":"2024-01-16T15:00:03.568172+0000","last_fullsized":"2024-01-16T15:00:03.568172+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:10:54.277062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.127245+0000","last_change":"2024-01-16T15:00:03.576575+0000","last_active":"2024-01-16T15:00:05.127245+0000","last_peered":"2024-01-16T15:00:05.127245+0000","last_clean":"2024-01-16T15:00:05.127245+0000","last_became_active":"2024-01-16T15:00:03.576266+0000","last_became_peered":"2024-01-16T15:00:03.576266+0000","last_unstale":"2024-01-16T15:00:05.127245+0000","last_undegraded":"2024-01-16T15:00:05.127245+0000","last_fullsized":"2024-01-16T15:00:05.127245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:28:03.212278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.584305+0000","last_change":"2024-01-16T15:00:02.636608+0000","last_active":"2024-01-16T15:00:03.584305+0000","last_peered":"2024-01-16T15:00:03.584305+0000","last_clean":"2024-01-16T15:00:03.584305+0000","last_became_active":"2024-01-16T15:00:02.636235+0000","last_became_peered":"2024-01-16T15:00:02.636235+0000","last_unstale":"2024-01-16T15:00:03.584305+0000","last_undegraded":"2024-01-16T15:00:03.584305+0000","last_fullsized":"2024-01-16T15:00:03.584305+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:11:10.313458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.553948+0000","last_change":"2024-01-16T15:00:03.579753+0000","last_active":"2024-01-16T15:00:04.553948+0000","last_peered":"2024-01-16T15:00:04.553948+0000","last_clean":"2024-01-16T15:00:04.553948+0000","last_became_active":"2024-01-16T15:00:03.579415+0000","last_became_peered":"2024-01-16T15:00:03.579415+0000","last_unstale":"2024-01-16T15:00:04.553948+0000","last_undegraded":"2024-01-16T15:00:04.553948+0000","last_fullsized":"2024-01-16T15:00:04.553948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:34:39.229613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.572357+0000","last_change":"2024-01-16T15:00:02.635549+0000","last_active":"2024-01-16T15:00:03.572357+0000","last_peered":"2024-01-16T15:00:03.572357+0000","last_clean":"2024-01-16T15:00:03.572357+0000","last_became_active":"2024-01-16T15:00:02.635309+0000","last_became_peered":"2024-01-16T15:00:02.635309+0000","last_unstale":"2024-01-16T15:00:03.572357+0000","last_undegraded":"2024-01-16T15:00:03.572357+0000","last_fullsized":"2024-01-16T15:00:03.572357+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:34:38.214930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323817+0000","last_change":"2024-01-16T15:00:03.586931+0000","last_active":"2024-01-16T15:00:05.323817+0000","last_peered":"2024-01-16T15:00:05.323817+0000","last_clean":"2024-01-16T15:00:05.323817+0000","last_became_active":"2024-01-16T15:00:03.586531+0000","last_became_peered":"2024-01-16T15:00:03.586531+0000","last_unstale":"2024-01-16T15:00:05.323817+0000","last_undegraded":"2024-01-16T15:00:05.323817+0000","last_fullsized":"2024-01-16T15:00:05.323817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:18:16.333187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.583406+0000","last_change":"2024-01-16T15:00:02.638637+0000","last_active":"2024-01-16T15:00:03.583406+0000","last_peered":"2024-01-16T15:00:03.583406+0000","last_clean":"2024-01-16T15:00:03.583406+0000","last_became_active":"2024-01-16T15:00:02.638355+0000","last_became_peered":"2024-01-16T15:00:02.638355+0000","last_unstale":"2024-01-16T15:00:03.583406+0000","last_undegraded":"2024-01-16T15:00:03.583406+0000","last_fullsized":"2024-01-16T15:00:03.583406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:53:48.731286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.324191+0000","last_change":"2024-01-16T15:00:03.587105+0000","last_active":"2024-01-16T15:00:05.324191+0000","last_peered":"2024-01-16T15:00:05.324191+0000","last_clean":"2024-01-16T15:00:05.324191+0000","last_became_active":"2024-01-16T15:00:03.586572+0000","last_became_peered":"2024-01-16T15:00:03.586572+0000","last_unstale":"2024-01-16T15:00:05.324191+0000","last_undegraded":"2024-01-16T15:00:05.324191+0000","last_fullsized":"2024-01-16T15:00:05.324191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:03:10.226190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.611665+0000","last_change":"2024-01-16T15:00:02.636352+0000","last_active":"2024-01-16T15:00:03.611665+0000","last_peered":"2024-01-16T15:00:03.611665+0000","last_clean":"2024-01-16T15:00:03.611665+0000","last_became_active":"2024-01-16T15:00:02.635069+0000","last_became_peered":"2024-01-16T15:00:02.635069+0000","last_unstale":"2024-01-16T15:00:03.611665+0000","last_undegraded":"2024-01-16T15:00:03.611665+0000","last_fullsized":"2024-01-16T15:00:03.611665+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:10:54.277062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.555540+0000","last_change":"2024-01-16T15:00:03.580722+0000","last_active":"2024-01-16T15:00:04.555540+0000","last_peered":"2024-01-16T15:00:04.555540+0000","last_clean":"2024-01-16T15:00:04.555540+0000","last_became_active":"2024-01-16T15:00:03.572307+0000","last_became_peered":"2024-01-16T15:00:03.572307+0000","last_unstale":"2024-01-16T15:00:04.555540+0000","last_undegraded":"2024-01-16T15:00:04.555540+0000","last_fullsized":"2024-01-16T15:00:04.555540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:25:28.863720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.617703+0000","last_change":"2024-01-16T15:00:02.625026+0000","last_active":"2024-01-16T15:00:03.617703+0000","last_peered":"2024-01-16T15:00:03.617703+0000","last_clean":"2024-01-16T15:00:03.617703+0000","last_became_active":"2024-01-16T15:00:02.622889+0000","last_became_peered":"2024-01-16T15:00:02.622889+0000","last_unstale":"2024-01-16T15:00:03.617703+0000","last_undegraded":"2024-01-16T15:00:03.617703+0000","last_fullsized":"2024-01-16T15:00:03.617703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:15:58.991139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.557063+0000","last_change":"2024-01-16T15:00:03.581190+0000","last_active":"2024-01-16T15:00:04.557063+0000","last_peered":"2024-01-16T15:00:04.557063+0000","last_clean":"2024-01-16T15:00:04.557063+0000","last_became_active":"2024-01-16T15:00:03.580903+0000","last_became_peered":"2024-01-16T15:00:03.580903+0000","last_unstale":"2024-01-16T15:00:04.557063+0000","last_undegraded":"2024-01-16T15:00:04.557063+0000","last_fullsized":"2024-01-16T15:00:04.557063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:17:11.329531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.615038+0000","last_change":"2024-01-16T15:00:02.622318+0000","last_active":"2024-01-16T15:00:03.615038+0000","last_peered":"2024-01-16T15:00:03.615038+0000","last_clean":"2024-01-16T15:00:03.615038+0000","last_became_active":"2024-01-16T15:00:02.622049+0000","last_became_peered":"2024-01-16T15:00:02.622049+0000","last_unstale":"2024-01-16T15:00:03.615038+0000","last_undegraded":"2024-01-16T15:00:03.615038+0000","last_fullsized":"2024-01-16T15:00:03.615038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:11:22.855828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323468+0000","last_change":"2024-01-16T15:00:03.574905+0000","last_active":"2024-01-16T15:00:05.323468+0000","last_peered":"2024-01-16T15:00:05.323468+0000","last_clean":"2024-01-16T15:00:05.323468+0000","last_became_active":"2024-01-16T15:00:03.572077+0000","last_became_peered":"2024-01-16T15:00:03.572077+0000","last_unstale":"2024-01-16T15:00:05.323468+0000","last_undegraded":"2024-01-16T15:00:05.323468+0000","last_fullsized":"2024-01-16T15:00:05.323468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:53:49.745969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.583051+0000","last_change":"2024-01-16T15:00:02.632349+0000","last_active":"2024-01-16T15:00:03.583051+0000","last_peered":"2024-01-16T15:00:03.583051+0000","last_clean":"2024-01-16T15:00:03.583051+0000","last_became_active":"2024-01-16T15:00:02.632038+0000","last_became_peered":"2024-01-16T15:00:02.632038+0000","last_unstale":"2024-01-16T15:00:03.583051+0000","last_undegraded":"2024-01-16T15:00:03.583051+0000","last_fullsized":"2024-01-16T15:00:03.583051+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:34:38.214930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.556735+0000","last_change":"2024-01-16T15:00:03.584616+0000","last_active":"2024-01-16T15:00:04.556735+0000","last_peered":"2024-01-16T15:00:04.556735+0000","last_clean":"2024-01-16T15:00:04.556735+0000","last_became_active":"2024-01-16T15:00:03.584348+0000","last_became_peered":"2024-01-16T15:00:03.584348+0000","last_unstale":"2024-01-16T15:00:04.556735+0000","last_undegraded":"2024-01-16T15:00:04.556735+0000","last_fullsized":"2024-01-16T15:00:04.556735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:18:16.333187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.569602+0000","last_change":"2024-01-16T15:00:02.628258+0000","last_active":"2024-01-16T15:00:03.569602+0000","last_peered":"2024-01-16T15:00:03.569602+0000","last_clean":"2024-01-16T15:00:03.569602+0000","last_became_active":"2024-01-16T15:00:02.627972+0000","last_became_peered":"2024-01-16T15:00:02.627972+0000","last_unstale":"2024-01-16T15:00:03.569602+0000","last_undegraded":"2024-01-16T15:00:03.569602+0000","last_fullsized":"2024-01-16T15:00:03.569602+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:25:27.849037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.750102+0000","last_change":"2024-01-16T15:00:03.588418+0000","last_active":"2024-01-16T15:00:04.750102+0000","last_peered":"2024-01-16T15:00:04.750102+0000","last_clean":"2024-01-16T15:00:04.750102+0000","last_became_active":"2024-01-16T15:00:03.587735+0000","last_became_peered":"2024-01-16T15:00:03.587735+0000","last_unstale":"2024-01-16T15:00:04.750102+0000","last_undegraded":"2024-01-16T15:00:04.750102+0000","last_fullsized":"2024-01-16T15:00:04.750102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:03:10.226190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.617567+0000","last_change":"2024-01-16T15:00:02.616381+0000","last_active":"2024-01-16T15:00:03.617567+0000","last_peered":"2024-01-16T15:00:03.617567+0000","last_clean":"2024-01-16T15:00:03.617567+0000","last_became_active":"2024-01-16T15:00:02.615244+0000","last_became_peered":"2024-01-16T15:00:02.615244+0000","last_unstale":"2024-01-16T15:00:03.617567+0000","last_undegraded":"2024-01-16T15:00:03.617567+0000","last_fullsized":"2024-01-16T15:00:03.617567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:21:47.228684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.556253+0000","last_change":"2024-01-16T15:00:03.584478+0000","last_active":"2024-01-16T15:00:04.556253+0000","last_peered":"2024-01-16T15:00:04.556253+0000","last_clean":"2024-01-16T15:00:04.556253+0000","last_became_active":"2024-01-16T15:00:03.583891+0000","last_became_peered":"2024-01-16T15:00:03.583891+0000","last_unstale":"2024-01-16T15:00:04.556253+0000","last_undegraded":"2024-01-16T15:00:04.556253+0000","last_fullsized":"2024-01-16T15:00:04.556253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:02:58.710795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.609993+0000","last_change":"2024-01-16T15:00:02.623236+0000","last_active":"2024-01-16T15:00:03.609993+0000","last_peered":"2024-01-16T15:00:03.609993+0000","last_clean":"2024-01-16T15:00:03.609993+0000","last_became_active":"2024-01-16T15:00:02.619961+0000","last_became_peered":"2024-01-16T15:00:02.619961+0000","last_unstale":"2024-01-16T15:00:03.609993+0000","last_undegraded":"2024-01-16T15:00:03.609993+0000","last_fullsized":"2024-01-16T15:00:03.609993+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:15:08.623284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.557756+0000","last_change":"2024-01-16T15:00:03.584135+0000","last_active":"2024-01-16T15:00:04.557756+0000","last_peered":"2024-01-16T15:00:04.557756+0000","last_clean":"2024-01-16T15:00:04.557756+0000","last_became_active":"2024-01-16T15:00:03.583742+0000","last_became_peered":"2024-01-16T15:00:03.583742+0000","last_unstale":"2024-01-16T15:00:04.557756+0000","last_undegraded":"2024-01-16T15:00:04.557756+0000","last_fullsized":"2024-01-16T15:00:04.557756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:54:11.712679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.613078+0000","last_change":"2024-01-16T15:00:02.625540+0000","last_active":"2024-01-16T15:00:03.613078+0000","last_peered":"2024-01-16T15:00:03.613078+0000","last_clean":"2024-01-16T15:00:03.613078+0000","last_became_active":"2024-01-16T15:00:02.625192+0000","last_became_peered":"2024-01-16T15:00:02.625192+0000","last_unstale":"2024-01-16T15:00:03.613078+0000","last_undegraded":"2024-01-16T15:00:03.613078+0000","last_fullsized":"2024-01-16T15:00:03.613078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:48:43.721848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.558230+0000","last_change":"2024-01-16T15:00:03.591505+0000","last_active":"2024-01-16T15:00:04.558230+0000","last_peered":"2024-01-16T15:00:04.558230+0000","last_clean":"2024-01-16T15:00:04.558230+0000","last_became_active":"2024-01-16T15:00:03.591187+0000","last_became_peered":"2024-01-16T15:00:03.591187+0000","last_unstale":"2024-01-16T15:00:04.558230+0000","last_undegraded":"2024-01-16T15:00:04.558230+0000","last_fullsized":"2024-01-16T15:00:04.558230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:05:33.003372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.610948+0000","last_change":"2024-01-16T15:00:02.627127+0000","last_active":"2024-01-16T15:00:03.610948+0000","last_peered":"2024-01-16T15:00:03.610948+0000","last_clean":"2024-01-16T15:00:03.610948+0000","last_became_active":"2024-01-16T15:00:02.624007+0000","last_became_peered":"2024-01-16T15:00:02.624007+0000","last_unstale":"2024-01-16T15:00:03.610948+0000","last_undegraded":"2024-01-16T15:00:03.610948+0000","last_fullsized":"2024-01-16T15:00:03.610948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:18:10.171130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323506+0000","last_change":"2024-01-16T15:00:03.588356+0000","last_active":"2024-01-16T15:00:05.323506+0000","last_peered":"2024-01-16T15:00:05.323506+0000","last_clean":"2024-01-16T15:00:05.323506+0000","last_became_active":"2024-01-16T15:00:03.588094+0000","last_became_peered":"2024-01-16T15:00:03.588094+0000","last_unstale":"2024-01-16T15:00:05.323506+0000","last_undegraded":"2024-01-16T15:00:05.323506+0000","last_fullsized":"2024-01-16T15:00:05.323506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:02:58.710795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.571707+0000","last_change":"2024-01-16T15:00:02.628534+0000","last_active":"2024-01-16T15:00:03.571707+0000","last_peered":"2024-01-16T15:00:03.571707+0000","last_clean":"2024-01-16T15:00:03.571707+0000","last_became_active":"2024-01-16T15:00:02.627252+0000","last_became_peered":"2024-01-16T15:00:02.627252+0000","last_unstale":"2024-01-16T15:00:03.571707+0000","last_undegraded":"2024-01-16T15:00:03.571707+0000","last_fullsized":"2024-01-16T15:00:03.571707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:17:47.960980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.556971+0000","last_change":"2024-01-16T15:00:03.588973+0000","last_active":"2024-01-16T15:00:04.556971+0000","last_peered":"2024-01-16T15:00:04.556971+0000","last_clean":"2024-01-16T15:00:04.556971+0000","last_became_active":"2024-01-16T15:00:03.588673+0000","last_became_peered":"2024-01-16T15:00:03.588673+0000","last_unstale":"2024-01-16T15:00:04.556971+0000","last_undegraded":"2024-01-16T15:00:04.556971+0000","last_fullsized":"2024-01-16T15:00:04.556971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:29:26.138536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.554975+0000","last_change":"2024-01-16T14:59:53.430715+0000","last_active":"2024-01-16T15:00:03.554975+0000","last_peered":"2024-01-16T15:00:03.554975+0000","last_clean":"2024-01-16T15:00:03.554975+0000","last_became_active":"2024-01-16T14:59:53.428209+0000","last_became_peered":"2024-01-16T14:59:53.428209+0000","last_unstale":"2024-01-16T15:00:03.554975+0000","last_undegraded":"2024-01-16T15:00:03.554975+0000","last_fullsized":"2024-01-16T15:00:03.554975+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:59:52.011232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:59:52.011232+0000","last_clean_scrub_stamp":"2024-01-16T14:59:52.011232+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:57:42.594848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.585945+0000","last_change":"2024-01-16T15:00:02.635313+0000","last_active":"2024-01-16T15:00:03.585945+0000","last_peered":"2024-01-16T15:00:03.585945+0000","last_clean":"2024-01-16T15:00:03.585945+0000","last_became_active":"2024-01-16T15:00:02.634804+0000","last_became_peered":"2024-01-16T15:00:02.634804+0000","last_unstale":"2024-01-16T15:00:03.585945+0000","last_undegraded":"2024-01-16T15:00:03.585945+0000","last_fullsized":"2024-01-16T15:00:03.585945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:28:02.197595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.127292+0000","last_change":"2024-01-16T15:00:03.590575+0000","last_active":"2024-01-16T15:00:05.127292+0000","last_peered":"2024-01-16T15:00:05.127292+0000","last_clean":"2024-01-16T15:00:05.127292+0000","last_became_active":"2024-01-16T15:00:03.590218+0000","last_became_peered":"2024-01-16T15:00:03.590218+0000","last_unstale":"2024-01-16T15:00:05.127292+0000","last_undegraded":"2024-01-16T15:00:05.127292+0000","last_fullsized":"2024-01-16T15:00:05.127292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:33:59.253631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.555195+0000","last_change":"2024-01-16T15:00:02.636675+0000","last_active":"2024-01-16T15:00:03.555195+0000","last_peered":"2024-01-16T15:00:03.555195+0000","last_clean":"2024-01-16T15:00:03.555195+0000","last_became_active":"2024-01-16T15:00:02.633102+0000","last_became_peered":"2024-01-16T15:00:02.633102+0000","last_unstale":"2024-01-16T15:00:03.555195+0000","last_undegraded":"2024-01-16T15:00:03.555195+0000","last_fullsized":"2024-01-16T15:00:03.555195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:57:13.913346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323042+0000","last_change":"2024-01-16T15:00:03.583866+0000","last_active":"2024-01-16T15:00:05.323042+0000","last_peered":"2024-01-16T15:00:05.323042+0000","last_clean":"2024-01-16T15:00:05.323042+0000","last_became_active":"2024-01-16T15:00:03.583561+0000","last_became_peered":"2024-01-16T15:00:03.583561+0000","last_unstale":"2024-01-16T15:00:05.323042+0000","last_undegraded":"2024-01-16T15:00:05.323042+0000","last_fullsized":"2024-01-16T15:00:05.323042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:25:28.863720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.571916+0000","last_change":"2024-01-16T15:00:02.624133+0000","last_active":"2024-01-16T15:00:03.571916+0000","last_peered":"2024-01-16T15:00:03.571916+0000","last_clean":"2024-01-16T15:00:03.571916+0000","last_became_active":"2024-01-16T15:00:02.623883+0000","last_became_peered":"2024-01-16T15:00:02.623883+0000","last_unstale":"2024-01-16T15:00:03.571916+0000","last_undegraded":"2024-01-16T15:00:03.571916+0000","last_fullsized":"2024-01-16T15:00:03.571916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:00:52.721183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.554313+0000","last_change":"2024-01-16T15:00:03.571151+0000","last_active":"2024-01-16T15:00:04.554313+0000","last_peered":"2024-01-16T15:00:04.554313+0000","last_clean":"2024-01-16T15:00:04.554313+0000","last_became_active":"2024-01-16T15:00:03.570800+0000","last_became_peered":"2024-01-16T15:00:03.570800+0000","last_unstale":"2024-01-16T15:00:04.554313+0000","last_undegraded":"2024-01-16T15:00:04.554313+0000","last_fullsized":"2024-01-16T15:00:04.554313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:03:10.226190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.586154+0000","last_change":"2024-01-16T15:00:02.637246+0000","last_active":"2024-01-16T15:00:03.586154+0000","last_peered":"2024-01-16T15:00:03.586154+0000","last_clean":"2024-01-16T15:00:03.586154+0000","last_became_active":"2024-01-16T15:00:02.636465+0000","last_became_peered":"2024-01-16T15:00:02.636465+0000","last_unstale":"2024-01-16T15:00:03.586154+0000","last_undegraded":"2024-01-16T15:00:03.586154+0000","last_fullsized":"2024-01-16T15:00:03.586154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:33:58.238948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.555119+0000","last_change":"2024-01-16T15:00:03.580644+0000","last_active":"2024-01-16T15:00:04.555119+0000","last_peered":"2024-01-16T15:00:04.555119+0000","last_clean":"2024-01-16T15:00:04.555119+0000","last_became_active":"2024-01-16T15:00:03.573120+0000","last_became_peered":"2024-01-16T15:00:03.573120+0000","last_unstale":"2024-01-16T15:00:04.555119+0000","last_undegraded":"2024-01-16T15:00:04.555119+0000","last_fullsized":"2024-01-16T15:00:04.555119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:21:48.243367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.563287+0000","last_change":"2024-01-16T15:00:02.620390+0000","last_active":"2024-01-16T15:00:03.563287+0000","last_peered":"2024-01-16T15:00:03.563287+0000","last_clean":"2024-01-16T15:00:03.563287+0000","last_became_active":"2024-01-16T15:00:02.619046+0000","last_became_peered":"2024-01-16T15:00:02.619046+0000","last_unstale":"2024-01-16T15:00:03.563287+0000","last_undegraded":"2024-01-16T15:00:03.563287+0000","last_fullsized":"2024-01-16T15:00:03.563287+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:48:43.721848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323450+0000","last_change":"2024-01-16T15:00:03.587239+0000","last_active":"2024-01-16T15:00:05.323450+0000","last_peered":"2024-01-16T15:00:05.323450+0000","last_clean":"2024-01-16T15:00:05.323450+0000","last_became_active":"2024-01-16T15:00:03.586925+0000","last_became_peered":"2024-01-16T15:00:03.586925+0000","last_unstale":"2024-01-16T15:00:05.323450+0000","last_undegraded":"2024-01-16T15:00:05.323450+0000","last_fullsized":"2024-01-16T15:00:05.323450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:09:47.428018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.555099+0000","last_change":"2024-01-16T15:00:02.640889+0000","last_active":"2024-01-16T15:00:03.555099+0000","last_peered":"2024-01-16T15:00:03.555099+0000","last_clean":"2024-01-16T15:00:03.555099+0000","last_became_active":"2024-01-16T15:00:02.633676+0000","last_became_peered":"2024-01-16T15:00:02.633676+0000","last_unstale":"2024-01-16T15:00:03.555099+0000","last_undegraded":"2024-01-16T15:00:03.555099+0000","last_fullsized":"2024-01-16T15:00:03.555099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:03:09.211508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.554126+0000","last_change":"2024-01-16T15:00:03.576742+0000","last_active":"2024-01-16T15:00:04.554126+0000","last_peered":"2024-01-16T15:00:04.554126+0000","last_clean":"2024-01-16T15:00:04.554126+0000","last_became_active":"2024-01-16T15:00:03.576104+0000","last_became_peered":"2024-01-16T15:00:03.576104+0000","last_unstale":"2024-01-16T15:00:04.554126+0000","last_undegraded":"2024-01-16T15:00:04.554126+0000","last_fullsized":"2024-01-16T15:00:04.554126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:28:03.212278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.554530+0000","last_change":"2024-01-16T15:00:02.633317+0000","last_active":"2024-01-16T15:00:04.554530+0000","last_peered":"2024-01-16T15:00:04.554530+0000","last_clean":"2024-01-16T15:00:04.554530+0000","last_became_active":"2024-01-16T15:00:02.632996+0000","last_became_peered":"2024-01-16T15:00:02.632996+0000","last_unstale":"2024-01-16T15:00:04.554530+0000","last_undegraded":"2024-01-16T15:00:04.554530+0000","last_fullsized":"2024-01-16T15:00:04.554530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:53:48.731286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.128153+0000","last_change":"2024-01-16T15:00:03.577601+0000","last_active":"2024-01-16T15:00:05.128153+0000","last_peered":"2024-01-16T15:00:05.128153+0000","last_clean":"2024-01-16T15:00:05.128153+0000","last_became_active":"2024-01-16T15:00:03.577107+0000","last_became_peered":"2024-01-16T15:00:03.577107+0000","last_unstale":"2024-01-16T15:00:05.128153+0000","last_undegraded":"2024-01-16T15:00:05.128153+0000","last_fullsized":"2024-01-16T15:00:05.128153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:31:09.422997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.612223+0000","last_change":"2024-01-16T15:00:02.634633+0000","last_active":"2024-01-16T15:00:03.612223+0000","last_peered":"2024-01-16T15:00:03.612223+0000","last_clean":"2024-01-16T15:00:03.612223+0000","last_became_active":"2024-01-16T15:00:02.634372+0000","last_became_peered":"2024-01-16T15:00:02.634372+0000","last_unstale":"2024-01-16T15:00:03.612223+0000","last_undegraded":"2024-01-16T15:00:03.612223+0000","last_fullsized":"2024-01-16T15:00:03.612223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:11:10.313458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.324092+0000","last_change":"2024-01-16T15:00:03.575017+0000","last_active":"2024-01-16T15:00:05.324092+0000","last_peered":"2024-01-16T15:00:05.324092+0000","last_clean":"2024-01-16T15:00:05.324092+0000","last_became_active":"2024-01-16T15:00:03.573750+0000","last_became_peered":"2024-01-16T15:00:03.573750+0000","last_unstale":"2024-01-16T15:00:05.324092+0000","last_undegraded":"2024-01-16T15:00:05.324092+0000","last_fullsized":"2024-01-16T15:00:05.324092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:28:03.212278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.552344+0000","last_change":"2024-01-16T15:00:02.636963+0000","last_active":"2024-01-16T15:00:03.552344+0000","last_peered":"2024-01-16T15:00:03.552344+0000","last_clean":"2024-01-16T15:00:03.552344+0000","last_became_active":"2024-01-16T15:00:02.632840+0000","last_became_peered":"2024-01-16T15:00:02.632840+0000","last_unstale":"2024-01-16T15:00:03.552344+0000","last_undegraded":"2024-01-16T15:00:03.552344+0000","last_fullsized":"2024-01-16T15:00:03.552344+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:21:47.228684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.554498+0000","last_change":"2024-01-16T15:00:03.582851+0000","last_active":"2024-01-16T15:00:04.554498+0000","last_peered":"2024-01-16T15:00:04.554498+0000","last_clean":"2024-01-16T15:00:04.554498+0000","last_became_active":"2024-01-16T15:00:03.581868+0000","last_became_peered":"2024-01-16T15:00:03.581868+0000","last_unstale":"2024-01-16T15:00:04.554498+0000","last_undegraded":"2024-01-16T15:00:04.554498+0000","last_fullsized":"2024-01-16T15:00:04.554498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:00:53.735866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.563577+0000","last_change":"2024-01-16T15:00:02.622029+0000","last_active":"2024-01-16T15:00:03.563577+0000","last_peered":"2024-01-16T15:00:03.563577+0000","last_clean":"2024-01-16T15:00:03.563577+0000","last_became_active":"2024-01-16T15:00:02.620427+0000","last_became_peered":"2024-01-16T15:00:02.620427+0000","last_unstale":"2024-01-16T15:00:03.563577+0000","last_undegraded":"2024-01-16T15:00:03.563577+0000","last_fullsized":"2024-01-16T15:00:03.563577+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:25:27.849037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.127523+0000","last_change":"2024-01-16T15:00:03.580221+0000","last_active":"2024-01-16T15:00:05.127523+0000","last_peered":"2024-01-16T15:00:05.127523+0000","last_clean":"2024-01-16T15:00:05.127523+0000","last_became_active":"2024-01-16T15:00:03.579960+0000","last_became_peered":"2024-01-16T15:00:03.579960+0000","last_unstale":"2024-01-16T15:00:05.127523+0000","last_undegraded":"2024-01-16T15:00:05.127523+0000","last_fullsized":"2024-01-16T15:00:05.127523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:03:10.226190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.564091+0000","last_change":"2024-01-16T15:00:02.631776+0000","last_active":"2024-01-16T15:00:03.564091+0000","last_peered":"2024-01-16T15:00:03.564091+0000","last_clean":"2024-01-16T15:00:03.564091+0000","last_became_active":"2024-01-16T15:00:02.630658+0000","last_became_peered":"2024-01-16T15:00:02.630658+0000","last_unstale":"2024-01-16T15:00:03.564091+0000","last_undegraded":"2024-01-16T15:00:03.564091+0000","last_fullsized":"2024-01-16T15:00:03.564091+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:15:58.991139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.128076+0000","last_change":"2024-01-16T15:00:03.574061+0000","last_active":"2024-01-16T15:00:05.128076+0000","last_peered":"2024-01-16T15:00:05.128076+0000","last_clean":"2024-01-16T15:00:05.128076+0000","last_became_active":"2024-01-16T15:00:03.573691+0000","last_became_peered":"2024-01-16T15:00:03.573691+0000","last_unstale":"2024-01-16T15:00:05.128076+0000","last_undegraded":"2024-01-16T15:00:05.128076+0000","last_fullsized":"2024-01-16T15:00:05.128076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:05:08.127179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.615335+0000","last_change":"2024-01-16T15:00:02.634122+0000","last_active":"2024-01-16T15:00:03.615335+0000","last_peered":"2024-01-16T15:00:03.615335+0000","last_clean":"2024-01-16T15:00:03.615335+0000","last_became_active":"2024-01-16T15:00:02.633819+0000","last_became_peered":"2024-01-16T15:00:02.633819+0000","last_unstale":"2024-01-16T15:00:03.615335+0000","last_undegraded":"2024-01-16T15:00:03.615335+0000","last_fullsized":"2024-01-16T15:00:03.615335+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:53:48.731286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:04.750129+0000","last_change":"2024-01-16T15:00:03.590510+0000","last_active":"2024-01-16T15:00:04.750129+0000","last_peered":"2024-01-16T15:00:04.750129+0000","last_clean":"2024-01-16T15:00:04.750129+0000","last_became_active":"2024-01-16T15:00:03.588555+0000","last_became_peered":"2024-01-16T15:00:03.588555+0000","last_unstale":"2024-01-16T15:00:04.750129+0000","last_undegraded":"2024-01-16T15:00:04.750129+0000","last_fullsized":"2024-01-16T15:00:04.750129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:06:24.858403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323387+0000","last_change":"2024-01-16T15:00:03.594894+0000","last_active":"2024-01-16T15:00:05.323387+0000","last_peered":"2024-01-16T15:00:05.323387+0000","last_clean":"2024-01-16T15:00:05.323387+0000","last_became_active":"2024-01-16T15:00:03.592616+0000","last_became_peered":"2024-01-16T15:00:03.592616+0000","last_unstale":"2024-01-16T15:00:05.323387+0000","last_undegraded":"2024-01-16T15:00:05.323387+0000","last_fullsized":"2024-01-16T15:00:05.323387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:31:09.422997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.615281+0000","last_change":"2024-01-16T15:00:02.631757+0000","last_active":"2024-01-16T15:00:03.615281+0000","last_peered":"2024-01-16T15:00:03.615281+0000","last_clean":"2024-01-16T15:00:03.615281+0000","last_became_active":"2024-01-16T15:00:02.630946+0000","last_became_peered":"2024-01-16T15:00:02.630946+0000","last_unstale":"2024-01-16T15:00:03.615281+0000","last_undegraded":"2024-01-16T15:00:03.615281+0000","last_fullsized":"2024-01-16T15:00:03.615281+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:34:38.214930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.323082+0000","last_change":"2024-01-16T15:00:03.593210+0000","last_active":"2024-01-16T15:00:05.323082+0000","last_peered":"2024-01-16T15:00:05.323082+0000","last_clean":"2024-01-16T15:00:05.323082+0000","last_became_active":"2024-01-16T15:00:03.592630+0000","last_became_peered":"2024-01-16T15:00:03.592630+0000","last_unstale":"2024-01-16T15:00:05.323082+0000","last_undegraded":"2024-01-16T15:00:05.323082+0000","last_fullsized":"2024-01-16T15:00:05.323082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:05:08.127179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.615363+0000","last_change":"2024-01-16T15:00:02.641351+0000","last_active":"2024-01-16T15:00:03.615363+0000","last_peered":"2024-01-16T15:00:03.615363+0000","last_clean":"2024-01-16T15:00:03.615363+0000","last_became_active":"2024-01-16T15:00:02.641021+0000","last_became_peered":"2024-01-16T15:00:02.641021+0000","last_unstale":"2024-01-16T15:00:03.615363+0000","last_undegraded":"2024-01-16T15:00:03.615363+0000","last_fullsized":"2024-01-16T15:00:03.615363+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:11:10.313458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.324164+0000","last_change":"2024-01-16T15:00:03.596490+0000","last_active":"2024-01-16T15:00:05.324164+0000","last_peered":"2024-01-16T15:00:05.324164+0000","last_clean":"2024-01-16T15:00:05.324164+0000","last_became_active":"2024-01-16T15:00:03.596137+0000","last_became_peered":"2024-01-16T15:00:03.596137+0000","last_unstale":"2024-01-16T15:00:05.324164+0000","last_undegraded":"2024-01-16T15:00:05.324164+0000","last_fullsized":"2024-01-16T15:00:05.324164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:33:59.253631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.612476+0000","last_change":"2024-01-16T15:00:02.630103+0000","last_active":"2024-01-16T15:00:03.612476+0000","last_peered":"2024-01-16T15:00:03.612476+0000","last_clean":"2024-01-16T15:00:03.612476+0000","last_became_active":"2024-01-16T15:00:02.629827+0000","last_became_peered":"2024-01-16T15:00:02.629827+0000","last_unstale":"2024-01-16T15:00:03.612476+0000","last_undegraded":"2024-01-16T15:00:03.612476+0000","last_fullsized":"2024-01-16T15:00:03.612476+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:25:27.849037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T15:00:05.127755+0000","last_change":"2024-01-16T15:00:03.583878+0000","last_active":"2024-01-16T15:00:05.127755+0000","last_peered":"2024-01-16T15:00:05.127755+0000","last_clean":"2024-01-16T15:00:05.127755+0000","last_became_active":"2024-01-16T15:00:03.583033+0000","last_became_peered":"2024-01-16T15:00:03.583033+0000","last_unstale":"2024-01-16T15:00:05.127755+0000","last_undegraded":"2024-01-16T15:00:05.127755+0000","last_fullsized":"2024-01-16T15:00:05.127755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:02.579838+0000","last_clean_scrub_stamp":"2024-01-16T15:00:02.579838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:57:14.928029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-16T15:00:03.616725+0000","last_change":"2024-01-16T15:00:02.634166+0000","last_active":"2024-01-16T15:00:03.616725+0000","last_peered":"2024-01-16T15:00:03.616725+0000","last_clean":"2024-01-16T15:00:03.616725+0000","last_became_active":"2024-01-16T15:00:02.633874+0000","last_became_peered":"2024-01-16T15:00:02.633874+0000","last_unstale":"2024-01-16T15:00:03.616725+0000","last_undegraded":"2024-01-16T15:00:03.616725+0000","last_fullsized":"2024-01-16T15:00:03.616725+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:00:01.565156+0000","last_clean_scrub_stamp":"2024-01-16T15:00:01.565156+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:39:13.413754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.093}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.264}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.514}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3720000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.204}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.242}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6779999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5660000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.518}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.179}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2170000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.901}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0049999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6619999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.615}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.327}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.909}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.27}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4670000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9950000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6509999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.952}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4350000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2889999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1359999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3740000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5640000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4670000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.024}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5019999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.407}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.3819999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3849999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":806912,"data_stored":660897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.427}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.268}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.216}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.74}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7929999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.321}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-16T15:00:13.906 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-01-16T15:00:13.906 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-01-16T15:00:13.906 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-01-16T15:00:13.906 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-01-16T15:00:14.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.168+0000 7fd4d8c00640 1 Processor -- start 2024-01-16T15:00:14.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.169+0000 7fd4d8c00640 1 -- start start 2024-01-16T15:00:14.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.170+0000 7fd4d8c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd4d40cf450 0x7fd4d40cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:14.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.170+0000 7fd4d8c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd4d40cdfb0 0x7fd4d40ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:14.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.170+0000 7fd4d8c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4d40cece0 con 0x7fd4d415c7f0 2024-01-16T15:00:14.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.170+0000 7fd4d8c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4d40cfe40 con 0x7fd4d40cf450 2024-01-16T15:00:14.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.170+0000 7fd4d8c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4d4156010 con 0x7fd4d40cdfb0 2024-01-16T15:00:14.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.170+0000 7fd4d1e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd4d415c7f0 0x7fd4d406db20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54222/0 (socket says 172.21.15.143:54222) 2024-01-16T15:00:14.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.170+0000 7fd4d1e00640 1 -- 172.21.15.143:0/1659039083 learned_addr learned my addr 172.21.15.143:0/1659039083 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:14.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.170+0000 7fd4c9200640 1 --2- 172.21.15.143:0/1659039083 >> v2:172.21.15.143:3300/0 conn(0x7fd4d40cf450 0x7fd4d40cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:14.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.170+0000 7fd4d2800640 1 --2- 172.21.15.143:0/1659039083 >> v2:172.21.15.204:3300/0 conn(0x7fd4d40cdfb0 0x7fd4d40ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:14.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.171+0000 7fd4d1400640 1 -- 172.21.15.143:0/1659039083 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3827650699 0 0) 0x7fd4d40cece0 con 0x7fd4d415c7f0 2024-01-16T15:00:14.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.171+0000 7fd4d1400640 1 -- 172.21.15.143:0/1659039083 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4b4003520 con 0x7fd4d415c7f0 2024-01-16T15:00:14.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.171+0000 7fd4c9200640 1 -- 172.21.15.143:0/1659039083 >> v2:172.21.15.204:3300/0 conn(0x7fd4d40cdfb0 msgr2=0x7fd4d40ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:14.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.171+0000 7fd4c9200640 1 --2- 172.21.15.143:0/1659039083 >> v2:172.21.15.204:3300/0 conn(0x7fd4d40cdfb0 0x7fd4d40ce3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:14.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.171+0000 7fd4c9200640 1 -- 172.21.15.143:0/1659039083 >> v1:172.21.15.143:6789/0 conn(0x7fd4d415c7f0 legacy=0x7fd4d406db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:14.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.171+0000 7fd4c9200640 1 -- 172.21.15.143:0/1659039083 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d406d040 con 0x7fd4d40cf450 2024-01-16T15:00:14.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.172+0000 7fd4c9200640 1 --2- 172.21.15.143:0/1659039083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4d40cf450 0x7fd4d40cf900 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd4d40d00e0 tx=0x7fd4bc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf8e1617b49e04e server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:14.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.172+0000 7fd4d1400640 1 -- 172.21.15.143:0/1659039083 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc02cc80 con 0x7fd4d40cf450 2024-01-16T15:00:14.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.173+0000 7fd4d1400640 1 -- 172.21.15.143:0/1659039083 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4bc02cde0 con 0x7fd4d40cf450 2024-01-16T15:00:14.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.173+0000 7fd4d1400640 1 -- 172.21.15.143:0/1659039083 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc034640 con 0x7fd4d40cf450 2024-01-16T15:00:14.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.173+0000 7fd4d8c00640 1 -- 172.21.15.143:0/1659039083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4d40cf450 msgr2=0x7fd4d40cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:14.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.173+0000 7fd4d8c00640 1 --2- 172.21.15.143:0/1659039083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4d40cf450 0x7fd4d40cf900 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd4d40d00e0 tx=0x7fd4bc02bed0 comp rx=0 tx=0).stop 2024-01-16T15:00:14.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.173+0000 7fd4d8c00640 1 -- 172.21.15.143:0/1659039083 shutdown_connections 2024-01-16T15:00:14.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.173+0000 7fd4d8c00640 1 --2- 172.21.15.143:0/1659039083 >> v2:172.21.15.204:3300/0 conn(0x7fd4d40cdfb0 0x7fd4d40ce3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:14.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.173+0000 7fd4d8c00640 1 --2- 172.21.15.143:0/1659039083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4d40cf450 0x7fd4d40cf900 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:14.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.173+0000 7fd4d8c00640 1 -- 172.21.15.143:0/1659039083 >> 172.21.15.143:0/1659039083 conn(0x7fd4d406a920 msgr2=0x7fd4d406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:14.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.173+0000 7fd4d8c00640 1 -- 172.21.15.143:0/1659039083 shutdown_connections 2024-01-16T15:00:14.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.173+0000 7fd4d8c00640 1 -- 172.21.15.143:0/1659039083 wait complete. 2024-01-16T15:00:14.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.175+0000 7fd4d8c00640 1 Processor -- start 2024-01-16T15:00:14.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.175+0000 7fd4d8c00640 1 -- start start 2024-01-16T15:00:14.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.176+0000 7fd4d8c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4d40cdfb0 0x7fd4d40ae140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:14.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.176+0000 7fd4d8c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4d40cf450 0x7fd4d40988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:14.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.176+0000 7fd4d8c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4d415c7f0 0x7fd4d415eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:14.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.176+0000 7fd4d8c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d41593e0 con 0x7fd4d40cf450 2024-01-16T15:00:14.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.176+0000 7fd4d8c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d4158f50 con 0x7fd4d40cdfb0 2024-01-16T15:00:14.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.176+0000 7fd4d8c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d40ae680 con 0x7fd4d415c7f0 2024-01-16T15:00:14.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.176+0000 7fd4c9200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4d40cf450 0x7fd4d40988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:14.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.177+0000 7fd4c9200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4d40cf450 0x7fd4d40988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42918/0 (socket says 172.21.15.143:42918) 2024-01-16T15:00:14.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.177+0000 7fd4c9200640 1 -- 172.21.15.143:0/513031165 learned_addr learned my addr 172.21.15.143:0/513031165 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:14.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.177+0000 7fd4d2800640 1 --2- 172.21.15.143:0/513031165 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4d415c7f0 0x7fd4d415eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:14.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.177+0000 7fd4d1e00640 1 --2- 172.21.15.143:0/513031165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4d40cdfb0 0x7fd4d40ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:14.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.177+0000 7fd4c9200640 1 -- 172.21.15.143:0/513031165 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4d415c7f0 msgr2=0x7fd4d415eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:14.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.177+0000 7fd4c9200640 1 --2- 172.21.15.143:0/513031165 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4d415c7f0 0x7fd4d415eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:14.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.177+0000 7fd4c9200640 1 -- 172.21.15.143:0/513031165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4d40cdfb0 msgr2=0x7fd4d40ae140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:14.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.177+0000 7fd4c9200640 1 --2- 172.21.15.143:0/513031165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4d40cdfb0 0x7fd4d40ae140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:14.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.177+0000 7fd4c9200640 1 -- 172.21.15.143:0/513031165 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4bc009a70 con 0x7fd4d40cf450 2024-01-16T15:00:14.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.177+0000 7fd4c9200640 1 --2- 172.21.15.143:0/513031165 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4d40cf450 0x7fd4d40988b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc02c4e0 tx=0x7fd4bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:14.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.178+0000 7fd4caa00640 1 -- 172.21.15.143:0/513031165 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc03e610 con 0x7fd4d40cf450 2024-01-16T15:00:14.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.178+0000 7fd4caa00640 1 -- 172.21.15.143:0/513031165 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4bc03e790 con 0x7fd4d40cf450 2024-01-16T15:00:14.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.178+0000 7fd4caa00640 1 -- 172.21.15.143:0/513031165 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc03eb40 con 0x7fd4d40cf450 2024-01-16T15:00:14.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.178+0000 7fd4d8c00640 1 -- 172.21.15.143:0/513031165 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4d41564c0 con 0x7fd4d40cf450 2024-01-16T15:00:14.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.178+0000 7fd4d8c00640 1 -- 172.21.15.143:0/513031165 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d4158900 con 0x7fd4d40cf450 2024-01-16T15:00:14.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.179+0000 7fd4d8c00640 1 -- 172.21.15.143:0/513031165 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4d41590e0 con 0x7fd4d40cf450 2024-01-16T15:00:14.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.180+0000 7fd4caa00640 1 -- 172.21.15.143:0/513031165 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fd4bc03b050 con 0x7fd4d40cf450 2024-01-16T15:00:14.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.181+0000 7fd4caa00640 1 --2- 172.21.15.143:0/513031165 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd4ac06b2c0 0x7fd4ac06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:14.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.181+0000 7fd4caa00640 1 -- 172.21.15.143:0/513031165 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd4bc0ac440 con 0x7fd4d40cf450 2024-01-16T15:00:14.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.181+0000 7fd4d1e00640 1 --2- 172.21.15.143:0/513031165 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd4ac06b2c0 0x7fd4ac06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:14.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.182+0000 7fd4d1e00640 1 --2- 172.21.15.143:0/513031165 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd4ac06b2c0 0x7fd4ac06d780 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4005880 tx=0x7fd4c4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:14.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.186+0000 7fd4caa00640 1 -- 172.21.15.143:0/513031165 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd4bc0772c0 con 0x7fd4d40cf450 2024-01-16T15:00:14.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.411+0000 7fd4d8c00640 1 -- 172.21.15.143:0/513031165 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd4d4067470 con 0x7fd4d40cf450 2024-01-16T15:00:14.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.412+0000 7fd4caa00640 1 -- 172.21.15.143:0/513031165 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fd4bc009450 con 0x7fd4d40cf450 2024-01-16T15:00:14.393 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:00:14.393 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-01-16T15:00:14.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.414+0000 7fd4d8c00640 1 -- 172.21.15.143:0/513031165 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd4ac06b2c0 msgr2=0x7fd4ac06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:14.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.414+0000 7fd4d8c00640 1 --2- 172.21.15.143:0/513031165 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd4ac06b2c0 0x7fd4ac06d780 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4005880 tx=0x7fd4c4009040 comp rx=0 tx=0).stop 2024-01-16T15:00:14.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.414+0000 7fd4d8c00640 1 -- 172.21.15.143:0/513031165 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4d40cf450 msgr2=0x7fd4d40988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:14.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.414+0000 7fd4d8c00640 1 --2- 172.21.15.143:0/513031165 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4d40cf450 0x7fd4d40988b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc02c4e0 tx=0x7fd4bc004a40 comp rx=0 tx=0).stop 2024-01-16T15:00:14.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.414+0000 7fd4d8c00640 1 -- 172.21.15.143:0/513031165 shutdown_connections 2024-01-16T15:00:14.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.414+0000 7fd4d8c00640 1 --2- 172.21.15.143:0/513031165 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd4ac06b2c0 0x7fd4ac06d780 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:14.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.414+0000 7fd4d8c00640 1 --2- 172.21.15.143:0/513031165 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4d415c7f0 0x7fd4d415eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:14.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.414+0000 7fd4d8c00640 1 --2- 172.21.15.143:0/513031165 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4d40cf450 0x7fd4d40988b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:14.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.414+0000 7fd4d8c00640 1 --2- 172.21.15.143:0/513031165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4d40cdfb0 0x7fd4d40ae140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:14.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.414+0000 7fd4d8c00640 1 -- 172.21.15.143:0/513031165 >> 172.21.15.143:0/513031165 conn(0x7fd4d406a920 msgr2=0x7fd4d4155570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:14.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.415+0000 7fd4d8c00640 1 -- 172.21.15.143:0/513031165 shutdown_connections 2024-01-16T15:00:14.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.415+0000 7fd4d8c00640 1 -- 172.21.15.143:0/513031165 wait complete. 2024-01-16T15:00:14.414 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-01-16T15:00:14.414 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T15:00:14.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.640+0000 7fe670c00640 1 Processor -- start 2024-01-16T15:00:14.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.640+0000 7fe670c00640 1 -- start start 2024-01-16T15:00:14.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.642+0000 7fe670c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe66c0b2250 0x7fe66c15df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:14.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.642+0000 7fe670c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe66c0b2630 0x7fe66c0ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:14.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.642+0000 7fe670c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe66c0cd470 0x7fe66c15b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:14.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.642+0000 7fe670c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe66c15a990 con 0x7fe66c0cd470 2024-01-16T15:00:14.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.642+0000 7fe670c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe66c15ab80 con 0x7fe66c0b2250 2024-01-16T15:00:14.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.642+0000 7fe670c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c15e680 con 0x7fe66c0b2630 2024-01-16T15:00:14.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.642+0000 7fe66a800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe66c0cd470 0x7fe66c15b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:14.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.642+0000 7fe66a800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe66c0cd470 0x7fe66c15b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42928/0 (socket says 172.21.15.143:42928) 2024-01-16T15:00:14.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.642+0000 7fe66a800640 1 -- 172.21.15.143:0/685839215 learned_addr learned my addr 172.21.15.143:0/685839215 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:14.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.642+0000 7fe669e00640 1 --2- 172.21.15.143:0/685839215 >> v2:172.21.15.204:3300/0 conn(0x7fe66c0b2250 0x7fe66c15df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:14.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.642+0000 7fe663e00640 1 --2- 172.21.15.143:0/685839215 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe66c0b2630 0x7fe66c0ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:14.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.642+0000 7fe66a800640 1 -- 172.21.15.143:0/685839215 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe66c0b2630 msgr2=0x7fe66c0ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:14.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.642+0000 7fe66a800640 1 --2- 172.21.15.143:0/685839215 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe66c0b2630 0x7fe66c0ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:14.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.642+0000 7fe66a800640 1 -- 172.21.15.143:0/685839215 >> v2:172.21.15.204:3300/0 conn(0x7fe66c0b2250 msgr2=0x7fe66c15df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:14.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.642+0000 7fe66a800640 1 --2- 172.21.15.143:0/685839215 >> v2:172.21.15.204:3300/0 conn(0x7fe66c0b2250 0x7fe66c15df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:14.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.642+0000 7fe66a800640 1 -- 172.21.15.143:0/685839215 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe66c15e960 con 0x7fe66c0cd470 2024-01-16T15:00:14.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.642+0000 7fe663e00640 1 --2- 172.21.15.143:0/685839215 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe66c0b2630 0x7fe66c0ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:14.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.643+0000 7fe66a800640 1 --2- 172.21.15.143:0/685839215 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe66c0cd470 0x7fe66c15b080 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe66c072a70 tx=0x7fe66402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83ae5caa49d419c1 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:14.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.644+0000 7fe669400640 1 -- 172.21.15.143:0/685839215 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66402ca80 con 0x7fe66c0cd470 2024-01-16T15:00:14.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.644+0000 7fe669400640 1 -- 172.21.15.143:0/685839215 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe66402cbe0 con 0x7fe66c0cd470 2024-01-16T15:00:14.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.644+0000 7fe669400640 1 -- 172.21.15.143:0/685839215 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6640343f0 con 0x7fe66c0cd470 2024-01-16T15:00:14.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.644+0000 7fe670c00640 1 -- 172.21.15.143:0/685839215 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe66c0cd470 msgr2=0x7fe66c15b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:14.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.644+0000 7fe670c00640 1 --2- 172.21.15.143:0/685839215 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe66c0cd470 0x7fe66c15b080 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe66c072a70 tx=0x7fe66402bee0 comp rx=0 tx=0).stop 2024-01-16T15:00:14.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.644+0000 7fe670c00640 1 -- 172.21.15.143:0/685839215 shutdown_connections 2024-01-16T15:00:14.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.644+0000 7fe670c00640 1 --2- 172.21.15.143:0/685839215 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe66c0cd470 0x7fe66c15b080 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:14.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.644+0000 7fe670c00640 1 --2- 172.21.15.143:0/685839215 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe66c0b2630 0x7fe66c0ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:14.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.644+0000 7fe670c00640 1 --2- 172.21.15.143:0/685839215 >> v2:172.21.15.204:3300/0 conn(0x7fe66c0b2250 0x7fe66c15df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:14.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.644+0000 7fe670c00640 1 -- 172.21.15.143:0/685839215 >> 172.21.15.143:0/685839215 conn(0x7fe66c06a920 msgr2=0x7fe66c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:14.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.644+0000 7fe670c00640 1 -- 172.21.15.143:0/685839215 shutdown_connections 2024-01-16T15:00:14.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.644+0000 7fe670c00640 1 -- 172.21.15.143:0/685839215 wait complete. 2024-01-16T15:00:14.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.646+0000 7fe670c00640 1 Processor -- start 2024-01-16T15:00:14.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.647+0000 7fe670c00640 1 -- start start 2024-01-16T15:00:14.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.647+0000 7fe670c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe66c0b2250 0x7fe66c14c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:14.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.647+0000 7fe670c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe66c0b2630 0x7fe66c145ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:14.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.648+0000 7fe670c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe66c0cd470 0x7fe66c14a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:14.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.648+0000 7fe670c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c146a70 con 0x7fe66c0b2630 2024-01-16T15:00:14.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.648+0000 7fe670c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c1f4cc0 con 0x7fe66c0cd470 2024-01-16T15:00:14.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.648+0000 7fe670c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c0cafd0 con 0x7fe66c0b2250 2024-01-16T15:00:14.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.648+0000 7fe663e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe66c0b2630 0x7fe66c145ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:14.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.648+0000 7fe663e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe66c0b2630 0x7fe66c145ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42944/0 (socket says 172.21.15.143:42944) 2024-01-16T15:00:14.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.648+0000 7fe663e00640 1 -- 172.21.15.143:0/3104379078 learned_addr learned my addr 172.21.15.143:0/3104379078 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:14.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.648+0000 7fe669e00640 1 --2- 172.21.15.143:0/3104379078 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe66c0b2250 0x7fe66c14c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:14.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.648+0000 7fe663e00640 1 -- 172.21.15.143:0/3104379078 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe66c0b2250 msgr2=0x7fe66c14c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:14.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.648+0000 7fe663e00640 1 --2- 172.21.15.143:0/3104379078 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe66c0b2250 0x7fe66c14c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:14.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.648+0000 7fe663e00640 1 -- 172.21.15.143:0/3104379078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe66c0cd470 msgr2=0x7fe66c14a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:14.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.648+0000 7fe66a800640 1 --2- 172.21.15.143:0/3104379078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe66c0cd470 0x7fe66c14a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:14.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.648+0000 7fe663e00640 1 --2- 172.21.15.143:0/3104379078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe66c0cd470 0x7fe66c14a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:14.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.648+0000 7fe663e00640 1 -- 172.21.15.143:0/3104379078 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe664009b30 con 0x7fe66c0b2630 2024-01-16T15:00:14.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.648+0000 7fe663e00640 1 --2- 172.21.15.143:0/3104379078 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe66c0b2630 0x7fe66c145ff0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe65400c820 tx=0x7fe65400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:14.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.649+0000 7fe662000640 1 -- 172.21.15.143:0/3104379078 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe654007a30 con 0x7fe66c0b2630 2024-01-16T15:00:14.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.649+0000 7fe662000640 1 -- 172.21.15.143:0/3104379078 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe654007b90 con 0x7fe66c0b2630 2024-01-16T15:00:14.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.649+0000 7fe662000640 1 -- 172.21.15.143:0/3104379078 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65400ce70 con 0x7fe66c0b2630 2024-01-16T15:00:14.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.649+0000 7fe670c00640 1 -- 172.21.15.143:0/3104379078 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe66c146530 con 0x7fe66c0b2630 2024-01-16T15:00:14.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.649+0000 7fe670c00640 1 -- 172.21.15.143:0/3104379078 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe66c0cc540 con 0x7fe66c0b2630 2024-01-16T15:00:14.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.650+0000 7fe670c00640 1 -- 172.21.15.143:0/3104379078 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe66c1f4e00 con 0x7fe66c0b2630 2024-01-16T15:00:14.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.651+0000 7fe662000640 1 -- 172.21.15.143:0/3104379078 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fe6540113f0 con 0x7fe66c0b2630 2024-01-16T15:00:14.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.652+0000 7fe662000640 1 --2- 172.21.15.143:0/3104379078 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe64406b2c0 0x7fe64406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:14.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.652+0000 7fe662000640 1 -- 172.21.15.143:0/3104379078 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe654089ce0 con 0x7fe66c0b2630 2024-01-16T15:00:14.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.652+0000 7fe669e00640 1 --2- 172.21.15.143:0/3104379078 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe64406b2c0 0x7fe64406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:14.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.653+0000 7fe669e00640 1 --2- 172.21.15.143:0/3104379078 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe64406b2c0 0x7fe64406d780 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe65c006fd0 tx=0x7fe65c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:14.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.657+0000 7fe662000640 1 -- 172.21.15.143:0/3104379078 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe66c1f4e00 con 0x7fe66c0b2630 2024-01-16T15:00:14.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.847+0000 7fe670c00640 1 -- 172.21.15.143:0/3104379078 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe66c1f4e00 con 0x7fe66c0b2630 2024-01-16T15:00:14.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.849+0000 7fe662000640 1 -- 172.21.15.143:0/3104379078 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16565 (secure 0 0 0) 0x7fe66c1f4e00 con 0x7fe66c0b2630 2024-01-16T15:00:14.831 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:00:14.831 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0","created":"2024-01-16T14:59:38.532601+0000","modified":"2024-01-16T15:00:04.543620+0000","last_up_change":"2024-01-16T14:59:50.983699+0000","last_in_change":"2024-01-16T14:59:42.056297+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:59:51.727720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T15:00:01.010472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T15:00:02.041228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f4c8891d-ab29-472a-907d-8446f20f7012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6801","nonce":1569293626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6805","nonce":1569293626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6814","nonce":1569293626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6809","nonce":1569293626}]},"public_addr":"172.21.15.143:6801/1569293626","cluster_addr":"172.21.15.143:6805/1569293626","heartbeat_back_addr":"172.21.15.143:6814/1569293626","heartbeat_front_addr":"172.21.15.143:6809/1569293626","state":["exists","up"]},{"osd":1,"uuid":"c4a2788d-8b89-4992-82b2-7f4b9debd509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6804","nonce":1276841168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6808","nonce":1276841168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6813","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6815","nonce":1276841168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6811","nonce":1276841168}]},"public_addr":"172.21.15.143:6804/1276841168","cluster_addr":"172.21.15.143:6808/1276841168","heartbeat_back_addr":"172.21.15.143:6815/1276841168","heartbeat_front_addr":"172.21.15.143:6811/1276841168","state":["exists","up"]},{"osd":2,"uuid":"64645c1d-31b2-4794-b0f2-7ccf8df30393","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6817","nonce":1217670170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6819","nonce":1217670170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6823","nonce":1217670170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6821","nonce":1217670170}]},"public_addr":"172.21.15.143:6817/1217670170","cluster_addr":"172.21.15.143:6819/1217670170","heartbeat_back_addr":"172.21.15.143:6823/1217670170","heartbeat_front_addr":"172.21.15.143:6821/1217670170","state":["exists","up"]},{"osd":3,"uuid":"73674129-a84e-4d31-8247-7d0179ae6df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6825","nonce":3414188012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6827","nonce":3414188012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6831","nonce":3414188012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6829","nonce":3414188012}]},"public_addr":"172.21.15.143:6825/3414188012","cluster_addr":"172.21.15.143:6827/3414188012","heartbeat_back_addr":"172.21.15.143:6831/3414188012","heartbeat_front_addr":"172.21.15.143:6829/3414188012","state":["exists","up"]},{"osd":4,"uuid":"cea28c9b-2f1c-4087-b415-9960789131d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6811","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6812","nonce":1070255716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6816","nonce":1070255716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6823","nonce":1070255716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6820","nonce":1070255716}]},"public_addr":"172.21.15.204:6812/1070255716","cluster_addr":"172.21.15.204:6816/1070255716","heartbeat_back_addr":"172.21.15.204:6823/1070255716","heartbeat_front_addr":"172.21.15.204:6820/1070255716","state":["exists","up"]},{"osd":5,"uuid":"33b63560-5574-4e78-983f-b398d67a2cb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6809","nonce":3605417837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6813","nonce":3605417837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6819","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6821","nonce":3605417837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6817","nonce":3605417837}]},"public_addr":"172.21.15.204:6809/3605417837","cluster_addr":"172.21.15.204:6813/3605417837","heartbeat_back_addr":"172.21.15.204:6821/3605417837","heartbeat_front_addr":"172.21.15.204:6817/3605417837","state":["exists","up"]},{"osd":6,"uuid":"df397427-8940-4eee-9c79-eece669d735d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6801","nonce":1085000534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6803","nonce":1085000534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6808","nonce":1085000534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6805","nonce":1085000534}]},"public_addr":"172.21.15.204:6801/1085000534","cluster_addr":"172.21.15.204:6803/1085000534","heartbeat_back_addr":"172.21.15.204:6808/1085000534","heartbeat_front_addr":"172.21.15.204:6805/1085000534","state":["exists","up"]},{"osd":7,"uuid":"a2e5a15c-8312-405a-ac02-f7e1fc9df2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6825","nonce":1337220609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6827","nonce":1337220609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6831","nonce":1337220609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6829","nonce":1337220609}]},"public_addr":"172.21.15.204:6825/1337220609","cluster_addr":"172.21.15.204:6827/1337220609","heartbeat_back_addr":"172.21.15.204:6831/1337220609","heartbeat_front_addr":"172.21.15.204:6829/1337220609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:49.984507+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.395306+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.632998+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.373627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.342920+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.438366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.331926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.406657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T15:00:14.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.852+0000 7fe670c00640 1 -- 172.21.15.143:0/3104379078 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe64406b2c0 msgr2=0x7fe64406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:14.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.852+0000 7fe670c00640 1 --2- 172.21.15.143:0/3104379078 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe64406b2c0 0x7fe64406d780 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe65c006fd0 tx=0x7fe65c008040 comp rx=0 tx=0).stop 2024-01-16T15:00:14.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.852+0000 7fe670c00640 1 -- 172.21.15.143:0/3104379078 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe66c0b2630 msgr2=0x7fe66c145ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:14.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.852+0000 7fe670c00640 1 --2- 172.21.15.143:0/3104379078 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe66c0b2630 0x7fe66c145ff0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe65400c820 tx=0x7fe65400cc20 comp rx=0 tx=0).stop 2024-01-16T15:00:14.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.852+0000 7fe670c00640 1 -- 172.21.15.143:0/3104379078 shutdown_connections 2024-01-16T15:00:14.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.852+0000 7fe670c00640 1 --2- 172.21.15.143:0/3104379078 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe64406b2c0 0x7fe64406d780 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:14.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.852+0000 7fe670c00640 1 --2- 172.21.15.143:0/3104379078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe66c0cd470 0x7fe66c14a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:14.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.852+0000 7fe670c00640 1 --2- 172.21.15.143:0/3104379078 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe66c0b2630 0x7fe66c145ff0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:14.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.852+0000 7fe670c00640 1 --2- 172.21.15.143:0/3104379078 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe66c0b2250 0x7fe66c14c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:14.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.852+0000 7fe670c00640 1 -- 172.21.15.143:0/3104379078 >> 172.21.15.143:0/3104379078 conn(0x7fe66c06a920 msgr2=0x7fe66c158590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:14.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.852+0000 7fe670c00640 1 -- 172.21.15.143:0/3104379078 shutdown_connections 2024-01-16T15:00:14.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:14.852+0000 7fe670c00640 1 -- 172.21.15.143:0/3104379078 wait complete. 2024-01-16T15:00:14.852 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-16T14:59:51.727720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-16T15:00:01.010472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-16T15:00:02.041228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-16T15:00:14.852 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-16T15:00:15.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.086+0000 7fb523c00640 1 Processor -- start 2024-01-16T15:00:15.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.086+0000 7fb523c00640 1 -- start start 2024-01-16T15:00:15.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.087+0000 7fb523c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb51c06de10 0x7fb51c15e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:15.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.087+0000 7fb523c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb51c15ec90 con 0x7fb51c072160 2024-01-16T15:00:15.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.087+0000 7fb523c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb51c0cd530 con 0x7fb51c0c49f0 2024-01-16T15:00:15.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.087+0000 7fb523c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb51c0c4ed0 con 0x7fb51c06de10 2024-01-16T15:00:15.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.088+0000 7fb521200640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb51c072160 0x7fb51c06d700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:37944/0 (socket says 172.21.15.143:37944) 2024-01-16T15:00:15.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.088+0000 7fb521200640 1 -- 172.21.15.143:0/2495119235 learned_addr learned my addr 172.21.15.143:0/2495119235 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:15.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.088+0000 7fb51b400640 1 -- 172.21.15.143:0/2495119235 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4127139599 0 0) 0x7fb50c001110 con 0x7fb51c072160 2024-01-16T15:00:15.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.088+0000 7fb51be00640 1 --2- 172.21.15.143:0/2495119235 >> v2:172.21.15.204:3300/0 conn(0x7fb51c06de10 0x7fb51c15e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:15.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.088+0000 7fb51b400640 1 -- 172.21.15.143:0/2495119235 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4fc003520 con 0x7fb51c072160 2024-01-16T15:00:15.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.089+0000 7fb51b400640 1 -- 172.21.15.143:0/2495119235 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4291764055 0 0) 0x7fb4fc003520 con 0x7fb51c072160 2024-01-16T15:00:15.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.089+0000 7fb51b400640 1 -- 172.21.15.143:0/2495119235 >> v2:172.21.15.204:3300/0 conn(0x7fb51c06de10 msgr2=0x7fb51c15e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:15.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.089+0000 7fb51b400640 1 --2- 172.21.15.143:0/2495119235 >> v2:172.21.15.204:3300/0 conn(0x7fb51c06de10 0x7fb51c15e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:15.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.089+0000 7fb51b400640 1 -- 172.21.15.143:0/2495119235 >> v1:172.21.15.204:6789/0 conn(0x7fb51c0c49f0 legacy=0x7fb51c0705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:15.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.089+0000 7fb51b400640 1 -- 172.21.15.143:0/2495119235 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb51c0c51b0 con 0x7fb51c072160 2024-01-16T15:00:15.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.089+0000 7fb51b400640 1 -- 172.21.15.143:0/2495119235 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7fb50c001800 con 0x7fb51c072160 2024-01-16T15:00:15.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.089+0000 7fb51b400640 1 -- 172.21.15.143:0/2495119235 >> v1:172.21.15.143:6789/0 conn(0x7fb51c072160 legacy=0x7fb51c06d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:15.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.089+0000 7fb51b400640 1 --2- 172.21.15.143:0/2495119235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4fc003970 0x7fb4fc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:15.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.089+0000 7fb51b400640 1 --2- 172.21.15.143:0/2495119235 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4fc0047c0 0x7fb4fc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:15.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.089+0000 7fb51b400640 1 --2- 172.21.15.143:0/2495119235 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4fc005180 0x7fb4fc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:15.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.089+0000 7fb51b400640 1 -- 172.21.15.143:0/2495119235 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c0c4ed0 con 0x7fb4fc005180 2024-01-16T15:00:15.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.089+0000 7fb51b400640 1 -- 172.21.15.143:0/2495119235 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc00c020 con 0x7fb4fc003970 2024-01-16T15:00:15.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.089+0000 7fb51b400640 1 -- 172.21.15.143:0/2495119235 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc00c1e0 con 0x7fb4fc0047c0 2024-01-16T15:00:15.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.089+0000 7fb521c00640 1 --2- 172.21.15.143:0/2495119235 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4fc005180 0x7fb4fc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:15.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.089+0000 7fb521200640 1 --2- 172.21.15.143:0/2495119235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4fc003970 0x7fb4fc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:15.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.089+0000 7fb51be00640 1 --2- 172.21.15.143:0/2495119235 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4fc0047c0 0x7fb4fc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:15.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.089+0000 7fb521c00640 1 -- 172.21.15.143:0/2495119235 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4fc0047c0 msgr2=0x7fb4fc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:15.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.089+0000 7fb521c00640 1 --2- 172.21.15.143:0/2495119235 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4fc0047c0 0x7fb4fc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:15.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.089+0000 7fb521c00640 1 -- 172.21.15.143:0/2495119235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4fc003970 msgr2=0x7fb4fc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:15.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.089+0000 7fb521c00640 1 --2- 172.21.15.143:0/2495119235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4fc003970 0x7fb4fc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:15.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.090+0000 7fb521c00640 1 -- 172.21.15.143:0/2495119235 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4fc003520 con 0x7fb4fc005180 2024-01-16T15:00:15.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.090+0000 7fb521c00640 1 -- 172.21.15.143:0/2495119235 reap_dead start 2024-01-16T15:00:15.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.091+0000 7fb521c00640 1 --2- 172.21.15.143:0/2495119235 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4fc005180 0x7fb4fc00b980 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb514001e20 tx=0x7fb4fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6730dde9d5b05416 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:15.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.091+0000 7fb51b400640 1 -- 172.21.15.143:0/2495119235 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb514003b70 con 0x7fb4fc005180 2024-01-16T15:00:15.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.091+0000 7fb51b400640 1 -- 172.21.15.143:0/2495119235 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb514003cd0 con 0x7fb4fc005180 2024-01-16T15:00:15.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.091+0000 7fb51b400640 1 -- 172.21.15.143:0/2495119235 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5140052f0 con 0x7fb4fc005180 2024-01-16T15:00:15.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.091+0000 7fb523c00640 1 -- 172.21.15.143:0/2495119235 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4fc005180 msgr2=0x7fb4fc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:15.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.091+0000 7fb523c00640 1 --2- 172.21.15.143:0/2495119235 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4fc005180 0x7fb4fc00b980 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb514001e20 tx=0x7fb4fc0057f0 comp rx=0 tx=0).stop 2024-01-16T15:00:15.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.091+0000 7fb523c00640 1 -- 172.21.15.143:0/2495119235 shutdown_connections 2024-01-16T15:00:15.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.091+0000 7fb523c00640 1 --2- 172.21.15.143:0/2495119235 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4fc005180 0x7fb4fc00b980 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:15.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.091+0000 7fb523c00640 1 -- 172.21.15.143:0/2495119235 >> 172.21.15.143:0/2495119235 conn(0x7fb51c0b0c50 msgr2=0x7fb51c0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:15.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.092+0000 7fb523c00640 1 -- 172.21.15.143:0/2495119235 shutdown_connections 2024-01-16T15:00:15.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.092+0000 7fb523c00640 1 -- 172.21.15.143:0/2495119235 wait complete. 2024-01-16T15:00:15.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.093+0000 7fb523c00640 1 Processor -- start 2024-01-16T15:00:15.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.093+0000 7fb523c00640 1 -- start start 2024-01-16T15:00:15.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.094+0000 7fb523c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4fc005180 0x7fb51c0c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:15.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.094+0000 7fb523c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb51c15b4c0 0x7fb51c158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:15.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.094+0000 7fb523c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb51c0ac210 0x7fb51c0ac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:15.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.094+0000 7fb523c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c0acbe0 con 0x7fb4fc005180 2024-01-16T15:00:15.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.094+0000 7fb523c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c15b950 con 0x7fb51c15b4c0 2024-01-16T15:00:15.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.094+0000 7fb523c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c15bc30 con 0x7fb51c0ac210 2024-01-16T15:00:15.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.095+0000 7fb521200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4fc005180 0x7fb51c0c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:15.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.095+0000 7fb521200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4fc005180 0x7fb51c0c7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45058/0 (socket says 172.21.15.143:45058) 2024-01-16T15:00:15.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.095+0000 7fb521200640 1 -- 172.21.15.143:0/2599279560 learned_addr learned my addr 172.21.15.143:0/2599279560 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:15.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.095+0000 7fb521200640 1 -- 172.21.15.143:0/2599279560 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb51c0ac210 msgr2=0x7fb51c0ac6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:15.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.095+0000 7fb521200640 1 --2- 172.21.15.143:0/2599279560 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb51c0ac210 0x7fb51c0ac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:15.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.095+0000 7fb521200640 1 -- 172.21.15.143:0/2599279560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb51c15b4c0 msgr2=0x7fb51c158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:15.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.095+0000 7fb521200640 1 --2- 172.21.15.143:0/2599279560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb51c15b4c0 0x7fb51c158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:15.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.095+0000 7fb521200640 1 -- 172.21.15.143:0/2599279560 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb51c15bf10 con 0x7fb4fc005180 2024-01-16T15:00:15.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.095+0000 7fb521200640 1 --2- 172.21.15.143:0/2599279560 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4fc005180 0x7fb51c0c7cb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb50c001930 tx=0x7fb50c007570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:15.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.095+0000 7fb518c00640 1 -- 172.21.15.143:0/2599279560 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c00f9c0 con 0x7fb4fc005180 2024-01-16T15:00:15.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.095+0000 7fb518c00640 1 -- 172.21.15.143:0/2599279560 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50c00fb20 con 0x7fb4fc005180 2024-01-16T15:00:15.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.095+0000 7fb518c00640 1 -- 172.21.15.143:0/2599279560 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c00feb0 con 0x7fb4fc005180 2024-01-16T15:00:15.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.096+0000 7fb523c00640 1 -- 172.21.15.143:0/2599279560 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb51c15c1f0 con 0x7fb4fc005180 2024-01-16T15:00:15.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.096+0000 7fb523c00640 1 -- 172.21.15.143:0/2599279560 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb51c0b2fa0 con 0x7fb4fc005180 2024-01-16T15:00:15.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.097+0000 7fb523c00640 1 -- 172.21.15.143:0/2599279560 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb51c0c60b0 con 0x7fb4fc005180 2024-01-16T15:00:15.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.098+0000 7fb518c00640 1 -- 172.21.15.143:0/2599279560 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fb50c01d460 con 0x7fb4fc005180 2024-01-16T15:00:15.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.099+0000 7fb518c00640 1 --2- 172.21.15.143:0/2599279560 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb4f406b2c0 0x7fb4f406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:15.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.099+0000 7fb518c00640 1 -- 172.21.15.143:0/2599279560 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb50c089c50 con 0x7fb4fc005180 2024-01-16T15:00:15.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.099+0000 7fb51be00640 1 --2- 172.21.15.143:0/2599279560 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb4f406b2c0 0x7fb4f406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:15.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.100+0000 7fb51be00640 1 --2- 172.21.15.143:0/2599279560 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb4f406b2c0 0x7fb4f406d780 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb51c09d280 tx=0x7fb504008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:15.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.103+0000 7fb518c00640 1 -- 172.21.15.143:0/2599279560 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb50c08f020 con 0x7fb4fc005180 2024-01-16T15:00:15.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.291+0000 7fb523c00640 1 -- 172.21.15.143:0/2599279560 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb51c15ec90 con 0x7fb4fc005180 2024-01-16T15:00:15.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.292+0000 7fb518c00640 1 -- 172.21.15.143:0/2599279560 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb51c15ec90 con 0x7fb4fc005180 2024-01-16T15:00:15.274 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-16T15:00:15.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.295+0000 7fb523c00640 1 -- 172.21.15.143:0/2599279560 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb4f406b2c0 msgr2=0x7fb4f406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:15.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.295+0000 7fb523c00640 1 --2- 172.21.15.143:0/2599279560 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb4f406b2c0 0x7fb4f406d780 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb51c09d280 tx=0x7fb504008040 comp rx=0 tx=0).stop 2024-01-16T15:00:15.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.295+0000 7fb523c00640 1 -- 172.21.15.143:0/2599279560 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4fc005180 msgr2=0x7fb51c0c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:15.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.295+0000 7fb523c00640 1 --2- 172.21.15.143:0/2599279560 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4fc005180 0x7fb51c0c7cb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb50c001930 tx=0x7fb50c007570 comp rx=0 tx=0).stop 2024-01-16T15:00:15.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.295+0000 7fb523c00640 1 -- 172.21.15.143:0/2599279560 shutdown_connections 2024-01-16T15:00:15.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.295+0000 7fb523c00640 1 --2- 172.21.15.143:0/2599279560 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb4f406b2c0 0x7fb4f406d780 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:15.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.295+0000 7fb523c00640 1 --2- 172.21.15.143:0/2599279560 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb51c0ac210 0x7fb51c0ac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:15.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.295+0000 7fb523c00640 1 --2- 172.21.15.143:0/2599279560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb51c15b4c0 0x7fb51c158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:15.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.295+0000 7fb523c00640 1 --2- 172.21.15.143:0/2599279560 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4fc005180 0x7fb51c0c7cb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:15.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.295+0000 7fb523c00640 1 -- 172.21.15.143:0/2599279560 >> 172.21.15.143:0/2599279560 conn(0x7fb51c0b0c50 msgr2=0x7fb51c1597c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:15.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.295+0000 7fb523c00640 1 -- 172.21.15.143:0/2599279560 shutdown_connections 2024-01-16T15:00:15.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.296+0000 7fb523c00640 1 -- 172.21.15.143:0/2599279560 wait complete. 2024-01-16T15:00:15.295 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-16T15:00:15.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.524+0000 7f6f8be00640 1 Processor -- start 2024-01-16T15:00:15.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.525+0000 7f6f8be00640 1 -- start start 2024-01-16T15:00:15.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.526+0000 7f6f8be00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6f840cb850 0x7f6f840cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:15.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.526+0000 7f6f8be00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6f840d0a00 0x7f6f8406ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:15.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.526+0000 7f6f8be00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f840cc170 con 0x7f6f840b40d0 2024-01-16T15:00:15.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.526+0000 7f6f8be00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f84070870 con 0x7f6f840d0a00 2024-01-16T15:00:15.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.526+0000 7f6f8be00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f84070b50 con 0x7f6f840cb850 2024-01-16T15:00:15.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.527+0000 7f6f89e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f6f840b40d0 0x7f6f840cf670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:37954/0 (socket says 172.21.15.143:37954) 2024-01-16T15:00:15.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.527+0000 7f6f89e00640 1 -- 172.21.15.143:0/4141321819 learned_addr learned my addr 172.21.15.143:0/4141321819 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:15.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.527+0000 7f6f89400640 1 --2- 172.21.15.143:0/4141321819 >> v2:172.21.15.204:3300/0 conn(0x7f6f840cb850 0x7f6f840cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:15.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.527+0000 7f6f88a00640 1 --2- 172.21.15.143:0/4141321819 >> v2:172.21.15.143:3300/0 conn(0x7f6f840d0a00 0x7f6f8406ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:15.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.527+0000 7f6f83e00640 1 -- 172.21.15.143:0/4141321819 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 608044006 0 0) 0x7f6f840cc170 con 0x7f6f840b40d0 2024-01-16T15:00:15.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.527+0000 7f6f88a00640 1 -- 172.21.15.143:0/4141321819 >> v2:172.21.15.204:3300/0 conn(0x7f6f840cb850 msgr2=0x7f6f840cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:15.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.527+0000 7f6f88a00640 1 --2- 172.21.15.143:0/4141321819 >> v2:172.21.15.204:3300/0 conn(0x7f6f840cb850 0x7f6f840cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:15.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.527+0000 7f6f88a00640 1 -- 172.21.15.143:0/4141321819 >> v1:172.21.15.143:6789/0 conn(0x7f6f840b40d0 legacy=0x7f6f840cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:15.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.527+0000 7f6f88a00640 1 -- 172.21.15.143:0/4141321819 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f84070e30 con 0x7f6f840d0a00 2024-01-16T15:00:15.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.528+0000 7f6f88a00640 1 --2- 172.21.15.143:0/4141321819 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f840d0a00 0x7f6f8406ff70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6f8406b440 tx=0x7f6f6c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0feeed51deacb50 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:15.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.529+0000 7f6f83e00640 1 -- 172.21.15.143:0/4141321819 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c02e070 con 0x7f6f840d0a00 2024-01-16T15:00:15.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.529+0000 7f6f83e00640 1 -- 172.21.15.143:0/4141321819 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6c0315a0 con 0x7f6f840d0a00 2024-01-16T15:00:15.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.529+0000 7f6f83e00640 1 -- 172.21.15.143:0/4141321819 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c031930 con 0x7f6f840d0a00 2024-01-16T15:00:15.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.529+0000 7f6f8be00640 1 -- 172.21.15.143:0/4141321819 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f840d0a00 msgr2=0x7f6f8406ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:15.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.529+0000 7f6f8be00640 1 --2- 172.21.15.143:0/4141321819 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f840d0a00 0x7f6f8406ff70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6f8406b440 tx=0x7f6f6c02b840 comp rx=0 tx=0).stop 2024-01-16T15:00:15.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.529+0000 7f6f8be00640 1 -- 172.21.15.143:0/4141321819 shutdown_connections 2024-01-16T15:00:15.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.529+0000 7f6f8be00640 1 --2- 172.21.15.143:0/4141321819 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f840d0a00 0x7f6f8406ff70 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:15.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.529+0000 7f6f8be00640 1 --2- 172.21.15.143:0/4141321819 >> v2:172.21.15.204:3300/0 conn(0x7f6f840cb850 0x7f6f840cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:15.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.529+0000 7f6f8be00640 1 -- 172.21.15.143:0/4141321819 >> 172.21.15.143:0/4141321819 conn(0x7f6f8406e7d0 msgr2=0x7f6f8406ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:15.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.529+0000 7f6f8be00640 1 -- 172.21.15.143:0/4141321819 shutdown_connections 2024-01-16T15:00:15.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.529+0000 7f6f8be00640 1 -- 172.21.15.143:0/4141321819 wait complete. 2024-01-16T15:00:15.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.531+0000 7f6f8be00640 1 Processor -- start 2024-01-16T15:00:15.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.531+0000 7f6f8be00640 1 -- start start 2024-01-16T15:00:15.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.532+0000 7f6f8be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f840b40d0 0x7f6f84157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:15.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.532+0000 7f6f8be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f840cb850 0x7f6f8415b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:15.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.532+0000 7f6f8be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f840d0a00 0x7f6f8415d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:15.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.532+0000 7f6f8be00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f84157bd0 con 0x7f6f840d0a00 2024-01-16T15:00:15.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.532+0000 7f6f8be00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8415c040 con 0x7f6f840cb850 2024-01-16T15:00:15.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.532+0000 7f6f8be00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8415c320 con 0x7f6f840b40d0 2024-01-16T15:00:15.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.532+0000 7f6f89e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f840d0a00 0x7f6f8415d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:15.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.532+0000 7f6f89e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f840d0a00 0x7f6f8415d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45076/0 (socket says 172.21.15.143:45076) 2024-01-16T15:00:15.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.532+0000 7f6f89e00640 1 -- 172.21.15.143:0/2725437908 learned_addr learned my addr 172.21.15.143:0/2725437908 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:15.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.533+0000 7f6f89400640 1 --2- 172.21.15.143:0/2725437908 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f840b40d0 0x7f6f84157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:15.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.533+0000 7f6f88a00640 1 --2- 172.21.15.143:0/2725437908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f840cb850 0x7f6f8415b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:15.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.533+0000 7f6f89e00640 1 -- 172.21.15.143:0/2725437908 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f840b40d0 msgr2=0x7f6f84157690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:15.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.533+0000 7f6f89e00640 1 --2- 172.21.15.143:0/2725437908 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f840b40d0 0x7f6f84157690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:15.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.533+0000 7f6f89e00640 1 -- 172.21.15.143:0/2725437908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f840cb850 msgr2=0x7f6f8415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:15.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.533+0000 7f6f89e00640 1 --2- 172.21.15.143:0/2725437908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f840cb850 0x7f6f8415b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:15.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.533+0000 7f6f89e00640 1 -- 172.21.15.143:0/2725437908 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f6c009400 con 0x7f6f840d0a00 2024-01-16T15:00:15.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.533+0000 7f6f89400640 1 --2- 172.21.15.143:0/2725437908 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f840b40d0 0x7f6f84157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:15.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.533+0000 7f6f89e00640 1 --2- 172.21.15.143:0/2725437908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f840d0a00 0x7f6f8415d670 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c00c780 tx=0x7f6f7c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:15.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.533+0000 7f6f81600640 1 -- 172.21.15.143:0/2725437908 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c00ce90 con 0x7f6f840d0a00 2024-01-16T15:00:15.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.533+0000 7f6f81600640 1 -- 172.21.15.143:0/2725437908 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f7c00a640 con 0x7f6f840d0a00 2024-01-16T15:00:15.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.533+0000 7f6f81600640 1 -- 172.21.15.143:0/2725437908 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c00a9d0 con 0x7f6f840d0a00 2024-01-16T15:00:15.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.533+0000 7f6f8be00640 1 -- 172.21.15.143:0/2725437908 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f8415c600 con 0x7f6f840d0a00 2024-01-16T15:00:15.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.534+0000 7f6f8be00640 1 -- 172.21.15.143:0/2725437908 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f8415cbc0 con 0x7f6f840d0a00 2024-01-16T15:00:15.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.535+0000 7f6f8be00640 1 -- 172.21.15.143:0/2725437908 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f8415cdb0 con 0x7f6f840d0a00 2024-01-16T15:00:15.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.535+0000 7f6f81600640 1 -- 172.21.15.143:0/2725437908 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f6f7c00ab30 con 0x7f6f840d0a00 2024-01-16T15:00:15.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.536+0000 7f6f81600640 1 --2- 172.21.15.143:0/2725437908 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6f5c06b2c0 0x7f6f5c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:15.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.536+0000 7f6f81600640 1 -- 172.21.15.143:0/2725437908 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f7c0890f0 con 0x7f6f840d0a00 2024-01-16T15:00:15.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.537+0000 7f6f89400640 1 --2- 172.21.15.143:0/2725437908 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6f5c06b2c0 0x7f6f5c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:15.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.537+0000 7f6f89400640 1 --2- 172.21.15.143:0/2725437908 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6f5c06b2c0 0x7f6f5c06d780 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6f74009a20 tx=0x7f6f74008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:15.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.540+0000 7f6f81600640 1 -- 172.21.15.143:0/2725437908 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6f7c053f60 con 0x7f6f840d0a00 2024-01-16T15:00:15.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.730+0000 7f6f8be00640 1 -- 172.21.15.143:0/2725437908 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6f8404e8b0 con 0x7f6f840d0a00 2024-01-16T15:00:15.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.731+0000 7f6f81600640 1 -- 172.21.15.143:0/2725437908 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6f7c018020 con 0x7f6f840d0a00 2024-01-16T15:00:15.712 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2024-01-16T15:00:15.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.734+0000 7f6f8be00640 1 -- 172.21.15.143:0/2725437908 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6f5c06b2c0 msgr2=0x7f6f5c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:15.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.734+0000 7f6f8be00640 1 --2- 172.21.15.143:0/2725437908 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6f5c06b2c0 0x7f6f5c06d780 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6f74009a20 tx=0x7f6f74008040 comp rx=0 tx=0).stop 2024-01-16T15:00:15.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.734+0000 7f6f8be00640 1 -- 172.21.15.143:0/2725437908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f840d0a00 msgr2=0x7f6f8415d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:15.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.734+0000 7f6f8be00640 1 --2- 172.21.15.143:0/2725437908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f840d0a00 0x7f6f8415d670 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c00c780 tx=0x7f6f7c00cc40 comp rx=0 tx=0).stop 2024-01-16T15:00:15.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.734+0000 7f6f8be00640 1 -- 172.21.15.143:0/2725437908 shutdown_connections 2024-01-16T15:00:15.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.734+0000 7f6f8be00640 1 --2- 172.21.15.143:0/2725437908 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6f5c06b2c0 0x7f6f5c06d780 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:15.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.734+0000 7f6f8be00640 1 --2- 172.21.15.143:0/2725437908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f840d0a00 0x7f6f8415d670 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:15.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.734+0000 7f6f8be00640 1 --2- 172.21.15.143:0/2725437908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f840cb850 0x7f6f8415b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:15.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.734+0000 7f6f8be00640 1 --2- 172.21.15.143:0/2725437908 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f840b40d0 0x7f6f84157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:15.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.734+0000 7f6f8be00640 1 -- 172.21.15.143:0/2725437908 >> 172.21.15.143:0/2725437908 conn(0x7f6f8406e7d0 msgr2=0x7f6f8406d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:15.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.734+0000 7f6f8be00640 1 -- 172.21.15.143:0/2725437908 shutdown_connections 2024-01-16T15:00:15.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.734+0000 7f6f8be00640 1 -- 172.21.15.143:0/2725437908 wait complete. 2024-01-16T15:00:15.733 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-16T15:00:15.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.962+0000 7fe17a200640 1 Processor -- start 2024-01-16T15:00:15.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.962+0000 7fe17a200640 1 -- start start 2024-01-16T15:00:15.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.964+0000 7fe17a200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe1740cc0f0 0x7fe1740c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:15.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.964+0000 7fe17a200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe1740c57e0 0x7fe17406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:15.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.964+0000 7fe17a200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe17415eea0 con 0x7fe17415b440 2024-01-16T15:00:15.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.964+0000 7fe17a200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe17415b820 con 0x7fe1740cc0f0 2024-01-16T15:00:15.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.964+0000 7fe17a200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1740abf30 con 0x7fe1740c57e0 2024-01-16T15:00:15.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.964+0000 7fe16be00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe1740cc0f0 0x7fe1740c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:15.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.964+0000 7fe16be00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe1740cc0f0 0x7fe1740c52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45092/0 (socket says 172.21.15.143:45092) 2024-01-16T15:00:15.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.964+0000 7fe16be00640 1 -- 172.21.15.143:0/3498080989 learned_addr learned my addr 172.21.15.143:0/3498080989 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:15.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.965+0000 7fe173e00640 1 --2- 172.21.15.143:0/3498080989 >> v2:172.21.15.204:3300/0 conn(0x7fe1740c57e0 0x7fe17406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:15.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.965+0000 7fe16be00640 1 -- 172.21.15.143:0/3498080989 >> v2:172.21.15.204:3300/0 conn(0x7fe1740c57e0 msgr2=0x7fe17406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:15.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.965+0000 7fe16be00640 1 --2- 172.21.15.143:0/3498080989 >> v2:172.21.15.204:3300/0 conn(0x7fe1740c57e0 0x7fe17406c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:15.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.965+0000 7fe16be00640 1 -- 172.21.15.143:0/3498080989 >> v1:172.21.15.204:6789/0 conn(0x7fe17415b440 legacy=0x7fe1740cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:15.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.965+0000 7fe16be00640 1 -- 172.21.15.143:0/3498080989 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe17415f0f0 con 0x7fe1740cc0f0 2024-01-16T15:00:15.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.965+0000 7fe173e00640 1 --2- 172.21.15.143:0/3498080989 >> v2:172.21.15.204:3300/0 conn(0x7fe1740c57e0 0x7fe17406c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:15.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.966+0000 7fe16be00640 1 --2- 172.21.15.143:0/3498080989 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1740cc0f0 0x7fe1740c52a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe1740c95e0 tx=0x7fe15c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef129fc9a34e7f server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:15.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.966+0000 7fe172a00640 1 -- 172.21.15.143:0/3498080989 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c0317e0 con 0x7fe1740cc0f0 2024-01-16T15:00:15.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.966+0000 7fe172a00640 1 -- 172.21.15.143:0/3498080989 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe15c031940 con 0x7fe1740cc0f0 2024-01-16T15:00:15.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.966+0000 7fe172a00640 1 -- 172.21.15.143:0/3498080989 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c031cd0 con 0x7fe1740cc0f0 2024-01-16T15:00:15.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.967+0000 7fe17a200640 1 -- 172.21.15.143:0/3498080989 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1740cc0f0 msgr2=0x7fe1740c52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:15.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.967+0000 7fe17a200640 1 --2- 172.21.15.143:0/3498080989 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1740cc0f0 0x7fe1740c52a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe1740c95e0 tx=0x7fe15c02b880 comp rx=0 tx=0).stop 2024-01-16T15:00:15.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.967+0000 7fe17a200640 1 -- 172.21.15.143:0/3498080989 shutdown_connections 2024-01-16T15:00:15.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.967+0000 7fe17a200640 1 --2- 172.21.15.143:0/3498080989 >> v2:172.21.15.204:3300/0 conn(0x7fe1740c57e0 0x7fe17406c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:15.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.967+0000 7fe17a200640 1 --2- 172.21.15.143:0/3498080989 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1740cc0f0 0x7fe1740c52a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:15.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.967+0000 7fe17a200640 1 -- 172.21.15.143:0/3498080989 >> 172.21.15.143:0/3498080989 conn(0x7fe1740b0c50 msgr2=0x7fe1740b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:15.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.967+0000 7fe17a200640 1 -- 172.21.15.143:0/3498080989 shutdown_connections 2024-01-16T15:00:15.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.967+0000 7fe17a200640 1 -- 172.21.15.143:0/3498080989 wait complete. 2024-01-16T15:00:15.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.969+0000 7fe17a200640 1 Processor -- start 2024-01-16T15:00:15.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.969+0000 7fe17a200640 1 -- start start 2024-01-16T15:00:15.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.970+0000 7fe17a200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1740c57e0 0x7fe1741554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:15.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.971+0000 7fe17a200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1740cc0f0 0x7fe1741528b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:15.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.971+0000 7fe17a200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe17415b440 0x7fe1740cde80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:15.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.971+0000 7fe17a200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe174152df0 con 0x7fe17415b440 2024-01-16T15:00:15.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.971+0000 7fe17a200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe174155a90 con 0x7fe1740c57e0 2024-01-16T15:00:15.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.971+0000 7fe17a200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe174155d70 con 0x7fe1740cc0f0 2024-01-16T15:00:15.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.971+0000 7fe173e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe17415b440 0x7fe1740cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:15.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.971+0000 7fe173e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe17415b440 0x7fe1740cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45106/0 (socket says 172.21.15.143:45106) 2024-01-16T15:00:15.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.971+0000 7fe173e00640 1 -- 172.21.15.143:0/1434476843 learned_addr learned my addr 172.21.15.143:0/1434476843 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:15.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.971+0000 7fe173e00640 1 -- 172.21.15.143:0/1434476843 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1740cc0f0 msgr2=0x7fe1741528b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:15.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.971+0000 7fe173e00640 1 --2- 172.21.15.143:0/1434476843 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1740cc0f0 0x7fe1741528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:15.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.971+0000 7fe173e00640 1 -- 172.21.15.143:0/1434476843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1740c57e0 msgr2=0x7fe1741554b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:15.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.971+0000 7fe173400640 1 --2- 172.21.15.143:0/1434476843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1740c57e0 0x7fe1741554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:15.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.971+0000 7fe173e00640 1 --2- 172.21.15.143:0/1434476843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1740c57e0 0x7fe1741554b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:15.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.971+0000 7fe173e00640 1 -- 172.21.15.143:0/1434476843 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe15c0094d0 con 0x7fe17415b440 2024-01-16T15:00:15.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.971+0000 7fe173e00640 1 --2- 172.21.15.143:0/1434476843 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe17415b440 0x7fe1740cde80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe16c00cef0 tx=0x7fe16c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:15.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.972+0000 7fe16b400640 1 -- 172.21.15.143:0/1434476843 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16c00d840 con 0x7fe17415b440 2024-01-16T15:00:15.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.972+0000 7fe16b400640 1 -- 172.21.15.143:0/1434476843 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe16c00d9c0 con 0x7fe17415b440 2024-01-16T15:00:15.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.972+0000 7fe16b400640 1 -- 172.21.15.143:0/1434476843 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16c00dd70 con 0x7fe17415b440 2024-01-16T15:00:15.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.972+0000 7fe17a200640 1 -- 172.21.15.143:0/1434476843 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe174156050 con 0x7fe17415b440 2024-01-16T15:00:15.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.972+0000 7fe17a200640 1 -- 172.21.15.143:0/1434476843 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1740ce3c0 con 0x7fe17415b440 2024-01-16T15:00:15.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.973+0000 7fe17a200640 1 -- 172.21.15.143:0/1434476843 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe17404e8b0 con 0x7fe17415b440 2024-01-16T15:00:15.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.974+0000 7fe16b400640 1 -- 172.21.15.143:0/1434476843 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fe16c01e3b0 con 0x7fe17415b440 2024-01-16T15:00:15.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.975+0000 7fe16b400640 1 --2- 172.21.15.143:0/1434476843 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe14c06b2c0 0x7fe14c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:15.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.975+0000 7fe16b400640 1 -- 172.21.15.143:0/1434476843 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe16c08a750 con 0x7fe17415b440 2024-01-16T15:00:15.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.975+0000 7fe173400640 1 --2- 172.21.15.143:0/1434476843 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe14c06b2c0 0x7fe14c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:15.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.976+0000 7fe173400640 1 --2- 172.21.15.143:0/1434476843 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe14c06b2c0 0x7fe14c06d780 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe164006fd0 tx=0x7fe164008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:15.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:15.980+0000 7fe16b400640 1 -- 172.21.15.143:0/1434476843 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe16c0555e0 con 0x7fe17415b440 2024-01-16T15:00:16.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.169+0000 7fe17a200640 1 -- 172.21.15.143:0/1434476843 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe17406dc80 con 0x7fe17415b440 2024-01-16T15:00:16.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.170+0000 7fe16b400640 1 -- 172.21.15.143:0/1434476843 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe16c01a020 con 0x7fe17415b440 2024-01-16T15:00:16.151 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2024-01-16T15:00:16.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.172+0000 7fe17a200640 1 -- 172.21.15.143:0/1434476843 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe14c06b2c0 msgr2=0x7fe14c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:16.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.172+0000 7fe17a200640 1 --2- 172.21.15.143:0/1434476843 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe14c06b2c0 0x7fe14c06d780 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe164006fd0 tx=0x7fe164008040 comp rx=0 tx=0).stop 2024-01-16T15:00:16.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.172+0000 7fe17a200640 1 -- 172.21.15.143:0/1434476843 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe17415b440 msgr2=0x7fe1740cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:16.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.172+0000 7fe17a200640 1 --2- 172.21.15.143:0/1434476843 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe17415b440 0x7fe1740cde80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe16c00cef0 tx=0x7fe16c00a710 comp rx=0 tx=0).stop 2024-01-16T15:00:16.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.173+0000 7fe17a200640 1 -- 172.21.15.143:0/1434476843 shutdown_connections 2024-01-16T15:00:16.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.173+0000 7fe17a200640 1 --2- 172.21.15.143:0/1434476843 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe14c06b2c0 0x7fe14c06d780 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:16.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.173+0000 7fe17a200640 1 --2- 172.21.15.143:0/1434476843 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe17415b440 0x7fe1740cde80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:16.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.173+0000 7fe17a200640 1 --2- 172.21.15.143:0/1434476843 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1740cc0f0 0x7fe1741528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:16.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.173+0000 7fe17a200640 1 --2- 172.21.15.143:0/1434476843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1740c57e0 0x7fe1741554b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:16.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.173+0000 7fe17a200640 1 -- 172.21.15.143:0/1434476843 >> 172.21.15.143:0/1434476843 conn(0x7fe1740b0c50 msgr2=0x7fe1740cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:16.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.173+0000 7fe17a200640 1 -- 172.21.15.143:0/1434476843 shutdown_connections 2024-01-16T15:00:16.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.173+0000 7fe17a200640 1 -- 172.21.15.143:0/1434476843 wait complete. 2024-01-16T15:00:16.171 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-16T15:00:16.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.393+0000 7f0fdd600640 1 Processor -- start 2024-01-16T15:00:16.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.393+0000 7f0fdd600640 1 -- start start 2024-01-16T15:00:16.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.394+0000 7f0fdd600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0fd815c7f0 0x7f0fd806db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:16.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.395+0000 7f0fdd600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fd80cece0 con 0x7f0fd80ce6c0 2024-01-16T15:00:16.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.395+0000 7f0fdd600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fd806d040 con 0x7f0fd80cf450 2024-01-16T15:00:16.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.395+0000 7f0fdd600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fd80ceaf0 con 0x7f0fd815c7f0 2024-01-16T15:00:16.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.395+0000 7f0fd6a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0fd815c7f0 0x7f0fd806db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:16.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.395+0000 7f0fd7400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f0fd80ce6c0 0x7f0fd80714d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:37956/0 (socket says 172.21.15.143:37956) 2024-01-16T15:00:16.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.395+0000 7f0fd7400640 1 -- 172.21.15.143:0/140198401 learned_addr learned my addr 172.21.15.143:0/140198401 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:16.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.395+0000 7f0fd6a00640 1 -- 172.21.15.143:0/140198401 >> v1:172.21.15.204:6789/0 conn(0x7f0fd80cf450 legacy=0x7f0fd80cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:16.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.395+0000 7f0fd6a00640 1 -- 172.21.15.143:0/140198401 >> v1:172.21.15.143:6789/0 conn(0x7f0fd80ce6c0 legacy=0x7f0fd80714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:16.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.395+0000 7f0fd6a00640 1 -- 172.21.15.143:0/140198401 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd806d320 con 0x7f0fd815c7f0 2024-01-16T15:00:16.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.396+0000 7f0fd6a00640 1 --2- 172.21.15.143:0/140198401 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fd815c7f0 0x7f0fd806db20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0fc80097a0 tx=0x7f0fc802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c19270d9e42a7f server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:16.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.397+0000 7f0fd6000640 1 -- 172.21.15.143:0/140198401 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc8031830 con 0x7f0fd815c7f0 2024-01-16T15:00:16.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.397+0000 7f0fd6000640 1 -- 172.21.15.143:0/140198401 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc8031990 con 0x7f0fd815c7f0 2024-01-16T15:00:16.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.397+0000 7f0fd6000640 1 -- 172.21.15.143:0/140198401 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc8031d20 con 0x7f0fd815c7f0 2024-01-16T15:00:16.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.397+0000 7f0fdd600640 1 -- 172.21.15.143:0/140198401 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fd815c7f0 msgr2=0x7f0fd806db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:16.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.397+0000 7f0fdd600640 1 --2- 172.21.15.143:0/140198401 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fd815c7f0 0x7f0fd806db20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0fc80097a0 tx=0x7f0fc802b8d0 comp rx=0 tx=0).stop 2024-01-16T15:00:16.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.398+0000 7f0fdd600640 1 -- 172.21.15.143:0/140198401 shutdown_connections 2024-01-16T15:00:16.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.398+0000 7f0fdd600640 1 --2- 172.21.15.143:0/140198401 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fd815c7f0 0x7f0fd806db20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:16.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.398+0000 7f0fdd600640 1 -- 172.21.15.143:0/140198401 >> 172.21.15.143:0/140198401 conn(0x7f0fd806a920 msgr2=0x7f0fd806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:16.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.398+0000 7f0fdd600640 1 -- 172.21.15.143:0/140198401 shutdown_connections 2024-01-16T15:00:16.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.398+0000 7f0fdd600640 1 -- 172.21.15.143:0/140198401 wait complete. 2024-01-16T15:00:16.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.400+0000 7f0fdd600640 1 Processor -- start 2024-01-16T15:00:16.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.400+0000 7f0fdd600640 1 -- start start 2024-01-16T15:00:16.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.401+0000 7f0fdd600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fd80ce6c0 0x7f0fd80add60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:16.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.401+0000 7f0fdd600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fd80cf450 0x7f0fd814ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:16.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.401+0000 7f0fdd600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fd815c7f0 0x7f0fd80b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:16.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.401+0000 7f0fdd600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd80abfc0 con 0x7f0fd815c7f0 2024-01-16T15:00:16.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.401+0000 7f0fdd600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd814f480 con 0x7f0fd80ce6c0 2024-01-16T15:00:16.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.401+0000 7f0fdd600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd80ae340 con 0x7f0fd80cf450 2024-01-16T15:00:16.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.401+0000 7f0fd7400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fd815c7f0 0x7f0fd80b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:16.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.401+0000 7f0fd7400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fd815c7f0 0x7f0fd80b3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45136/0 (socket says 172.21.15.143:45136) 2024-01-16T15:00:16.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.401+0000 7f0fd7400640 1 -- 172.21.15.143:0/3108746353 learned_addr learned my addr 172.21.15.143:0/3108746353 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:16.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.401+0000 7f0fd6a00640 1 --2- 172.21.15.143:0/3108746353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fd80ce6c0 0x7f0fd80add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:16.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.401+0000 7f0fcfe00640 1 --2- 172.21.15.143:0/3108746353 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fd80cf450 0x7f0fd814ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:16.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.402+0000 7f0fd7400640 1 -- 172.21.15.143:0/3108746353 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fd80cf450 msgr2=0x7f0fd814ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:16.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.402+0000 7f0fd7400640 1 --2- 172.21.15.143:0/3108746353 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fd80cf450 0x7f0fd814ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:16.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.402+0000 7f0fd7400640 1 -- 172.21.15.143:0/3108746353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fd80ce6c0 msgr2=0x7f0fd80add60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:16.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.402+0000 7f0fd7400640 1 --2- 172.21.15.143:0/3108746353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fd80ce6c0 0x7f0fd80add60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:16.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.402+0000 7f0fd7400640 1 -- 172.21.15.143:0/3108746353 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc8009400 con 0x7f0fd815c7f0 2024-01-16T15:00:16.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.402+0000 7f0fd6a00640 1 --2- 172.21.15.143:0/3108746353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fd80ce6c0 0x7f0fd80add60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T15:00:16.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.402+0000 7f0fd7400640 1 --2- 172.21.15.143:0/3108746353 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fd815c7f0 0x7f0fd80b3b30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000caa0 tx=0x7f0fd000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:16.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.402+0000 7f0fcea00640 1 -- 172.21.15.143:0/3108746353 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd000ac00 con 0x7f0fd815c7f0 2024-01-16T15:00:16.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.402+0000 7f0fcea00640 1 -- 172.21.15.143:0/3108746353 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd000ad60 con 0x7f0fd815c7f0 2024-01-16T15:00:16.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.402+0000 7f0fcea00640 1 -- 172.21.15.143:0/3108746353 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd0012600 con 0x7f0fd815c7f0 2024-01-16T15:00:16.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.402+0000 7f0fdd600640 1 -- 172.21.15.143:0/3108746353 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fd80ae620 con 0x7f0fd815c7f0 2024-01-16T15:00:16.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.403+0000 7f0fdd600640 1 -- 172.21.15.143:0/3108746353 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fd80ac4f0 con 0x7f0fd815c7f0 2024-01-16T15:00:16.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.404+0000 7f0fdd600640 1 -- 172.21.15.143:0/3108746353 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fd80ac730 con 0x7f0fd815c7f0 2024-01-16T15:00:16.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.404+0000 7f0fcea00640 1 -- 172.21.15.143:0/3108746353 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f0fd0007070 con 0x7f0fd815c7f0 2024-01-16T15:00:16.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.405+0000 7f0fcea00640 1 --2- 172.21.15.143:0/3108746353 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0fb006b2c0 0x7f0fb006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:16.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.405+0000 7f0fcea00640 1 -- 172.21.15.143:0/3108746353 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0fd00899b0 con 0x7f0fd815c7f0 2024-01-16T15:00:16.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.406+0000 7f0fd6a00640 1 --2- 172.21.15.143:0/3108746353 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0fb006b2c0 0x7f0fb006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:16.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.406+0000 7f0fd6a00640 1 --2- 172.21.15.143:0/3108746353 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0fb006b2c0 0x7f0fb006d780 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8036000 tx=0x7f0fc803c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:16.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.410+0000 7f0fcea00640 1 -- 172.21.15.143:0/3108746353 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0fd80ac730 con 0x7f0fd815c7f0 2024-01-16T15:00:16.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.638+0000 7f0fdd600640 1 -- 172.21.15.143:0/3108746353 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0fd80ac730 con 0x7f0fd815c7f0 2024-01-16T15:00:16.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.639+0000 7f0fcea00640 1 -- 172.21.15.143:0/3108746353 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4916 (secure 0 0 0) 0x7f0fd80ac730 con 0x7f0fd815c7f0 2024-01-16T15:00:16.620 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:00:16.620 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3599169408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3599169408},{"type":"v1","addr":"172.21.15.204:6834","nonce":3599169408}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/859082411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":859082411},{"type":"v1","addr":"172.21.15.204:6835","nonce":859082411}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6836/1564738611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1564738611},{"type":"v1","addr":"172.21.15.143:6836","nonce":1564738611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-16T15:00:03.050082+0000","modified":"2024-01-16T15:00:06.109664+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/3120706283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6835","nonce":3120706283},{"type":"v1","addr":"172.21.15.143:6837","nonce":3120706283}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-16T15:00:16.620 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 8 2024-01-16T15:00:16.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.641+0000 7f0fdd600640 1 -- 172.21.15.143:0/3108746353 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0fb006b2c0 msgr2=0x7f0fb006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:16.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.641+0000 7f0fdd600640 1 --2- 172.21.15.143:0/3108746353 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0fb006b2c0 0x7f0fb006d780 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8036000 tx=0x7f0fc803c040 comp rx=0 tx=0).stop 2024-01-16T15:00:16.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.641+0000 7f0fdd600640 1 -- 172.21.15.143:0/3108746353 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fd815c7f0 msgr2=0x7f0fd80b3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:16.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.641+0000 7f0fdd600640 1 --2- 172.21.15.143:0/3108746353 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fd815c7f0 0x7f0fd80b3b30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000caa0 tx=0x7f0fd000cf60 comp rx=0 tx=0).stop 2024-01-16T15:00:16.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.642+0000 7f0fdd600640 1 -- 172.21.15.143:0/3108746353 shutdown_connections 2024-01-16T15:00:16.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.642+0000 7f0fdd600640 1 --2- 172.21.15.143:0/3108746353 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0fb006b2c0 0x7f0fb006d780 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:16.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.642+0000 7f0fdd600640 1 --2- 172.21.15.143:0/3108746353 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fd815c7f0 0x7f0fd80b3b30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:16.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.642+0000 7f0fdd600640 1 --2- 172.21.15.143:0/3108746353 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fd80cf450 0x7f0fd814ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:16.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.642+0000 7f0fdd600640 1 --2- 172.21.15.143:0/3108746353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fd80ce6c0 0x7f0fd80add60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:16.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.642+0000 7f0fdd600640 1 -- 172.21.15.143:0/3108746353 >> 172.21.15.143:0/3108746353 conn(0x7f0fd806a920 msgr2=0x7f0fd80af9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:16.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.642+0000 7f0fdd600640 1 -- 172.21.15.143:0/3108746353 shutdown_connections 2024-01-16T15:00:16.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.642+0000 7f0fdd600640 1 -- 172.21.15.143:0/3108746353 wait complete. 2024-01-16T15:00:16.642 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T15:00:16.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.867+0000 7fbc1e200640 1 Processor -- start 2024-01-16T15:00:16.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.868+0000 7fbc1e200640 1 -- start start 2024-01-16T15:00:16.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.869+0000 7fbc1e200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbc180cc470 0x7fbc180abfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:16.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.869+0000 7fbc1e200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc180ce270 con 0x7fbc180cfb60 2024-01-16T15:00:16.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.869+0000 7fbc1e200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc1806d0a0 con 0x7fbc180ce640 2024-01-16T15:00:16.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.869+0000 7fbc1e200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc1806c560 con 0x7fbc180cc470 2024-01-16T15:00:16.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.869+0000 7fbc17e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fbc180cfb60 0x7fbc1806d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:37968/0 (socket says 172.21.15.143:37968) 2024-01-16T15:00:16.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.869+0000 7fbc17e00640 1 -- 172.21.15.143:0/603856416 learned_addr learned my addr 172.21.15.143:0/603856416 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:16.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.870+0000 7fbc17400640 1 --2- 172.21.15.143:0/603856416 >> v2:172.21.15.204:3300/0 conn(0x7fbc180cc470 0x7fbc180abfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:16.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.870+0000 7fbc16000640 1 -- 172.21.15.143:0/603856416 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1308926145 0 0) 0x7fbc180ce270 con 0x7fbc180cfb60 2024-01-16T15:00:16.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.870+0000 7fbc16000640 1 -- 172.21.15.143:0/603856416 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbf8003520 con 0x7fbc180cfb60 2024-01-16T15:00:16.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.870+0000 7fbc16000640 1 -- 172.21.15.143:0/603856416 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4266863139 0 0) 0x7fbbf8003520 con 0x7fbc180cfb60 2024-01-16T15:00:16.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.870+0000 7fbc16000640 1 -- 172.21.15.143:0/603856416 >> v2:172.21.15.204:3300/0 conn(0x7fbc180cc470 msgr2=0x7fbc180abfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:16.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.870+0000 7fbc16000640 1 --2- 172.21.15.143:0/603856416 >> v2:172.21.15.204:3300/0 conn(0x7fbc180cc470 0x7fbc180abfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:16.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.870+0000 7fbc16000640 1 -- 172.21.15.143:0/603856416 >> v1:172.21.15.204:6789/0 conn(0x7fbc180ce640 legacy=0x7fbc180cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:16.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.870+0000 7fbc16000640 1 -- 172.21.15.143:0/603856416 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc1806c770 con 0x7fbc180cfb60 2024-01-16T15:00:16.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.870+0000 7fbc16000640 1 -- 172.21.15.143:0/603856416 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7fbc100019f0 con 0x7fbc180cfb60 2024-01-16T15:00:16.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.870+0000 7fbc16000640 1 -- 172.21.15.143:0/603856416 >> v1:172.21.15.143:6789/0 conn(0x7fbc180cfb60 legacy=0x7fbc1806d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:16.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.870+0000 7fbc16000640 1 --2- 172.21.15.143:0/603856416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbf8003970 0x7fbbf8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:16.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.870+0000 7fbc16000640 1 --2- 172.21.15.143:0/603856416 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbbf80047c0 0x7fbbf8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:16.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.870+0000 7fbc16000640 1 --2- 172.21.15.143:0/603856416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbf8005180 0x7fbbf800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:16.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.870+0000 7fbc16000640 1 -- 172.21.15.143:0/603856416 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1806c560 con 0x7fbbf8003970 2024-01-16T15:00:16.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.870+0000 7fbc16000640 1 -- 172.21.15.143:0/603856416 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf800c0d0 con 0x7fbbf8005180 2024-01-16T15:00:16.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.870+0000 7fbc16000640 1 -- 172.21.15.143:0/603856416 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf800c290 con 0x7fbbf80047c0 2024-01-16T15:00:16.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.871+0000 7fbc17400640 1 --2- 172.21.15.143:0/603856416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbf8003970 0x7fbbf8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:16.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.871+0000 7fbc16a00640 1 --2- 172.21.15.143:0/603856416 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbbf80047c0 0x7fbbf8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:16.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.871+0000 7fbc17e00640 1 --2- 172.21.15.143:0/603856416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbf8005180 0x7fbbf800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:16.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.871+0000 7fbc17400640 1 -- 172.21.15.143:0/603856416 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbbf80047c0 msgr2=0x7fbbf8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:16.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.871+0000 7fbc17400640 1 --2- 172.21.15.143:0/603856416 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbbf80047c0 0x7fbbf8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:16.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.871+0000 7fbc17400640 1 -- 172.21.15.143:0/603856416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbf8005180 msgr2=0x7fbbf800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:16.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.871+0000 7fbc17400640 1 --2- 172.21.15.143:0/603856416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbf8005180 0x7fbbf800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:16.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.871+0000 7fbc17400640 1 -- 172.21.15.143:0/603856416 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf8003520 con 0x7fbbf8003970 2024-01-16T15:00:16.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.871+0000 7fbc16a00640 1 --2- 172.21.15.143:0/603856416 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbbf80047c0 0x7fbbf8004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:16.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.871+0000 7fbc17e00640 1 --2- 172.21.15.143:0/603856416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbf8005180 0x7fbbf800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:16.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.871+0000 7fbc17e00640 1 -- 172.21.15.143:0/603856416 reap_dead start 2024-01-16T15:00:16.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.872+0000 7fbc17400640 1 --2- 172.21.15.143:0/603856416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbf8003970 0x7fbbf8003d90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbc0800ba30 tx=0x7fbbf80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db99f52142fa57e server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:16.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.872+0000 7fbc16000640 1 -- 172.21.15.143:0/603856416 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc08032640 con 0x7fbbf8003970 2024-01-16T15:00:16.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.872+0000 7fbc16000640 1 -- 172.21.15.143:0/603856416 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc080327a0 con 0x7fbbf8003970 2024-01-16T15:00:16.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.872+0000 7fbc16000640 1 -- 172.21.15.143:0/603856416 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc08032b30 con 0x7fbbf8003970 2024-01-16T15:00:16.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.873+0000 7fbc1e200640 1 -- 172.21.15.143:0/603856416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbf8003970 msgr2=0x7fbbf8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:16.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.873+0000 7fbc1e200640 1 --2- 172.21.15.143:0/603856416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbf8003970 0x7fbbf8003d90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbc0800ba30 tx=0x7fbbf80057f0 comp rx=0 tx=0).stop 2024-01-16T15:00:16.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.873+0000 7fbc1e200640 1 -- 172.21.15.143:0/603856416 shutdown_connections 2024-01-16T15:00:16.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.873+0000 7fbc1e200640 1 --2- 172.21.15.143:0/603856416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbf8003970 0x7fbbf8003d90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:16.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.873+0000 7fbc1e200640 1 -- 172.21.15.143:0/603856416 >> 172.21.15.143:0/603856416 conn(0x7fbc1806a920 msgr2=0x7fbc1806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:16.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.873+0000 7fbc1e200640 1 -- 172.21.15.143:0/603856416 shutdown_connections 2024-01-16T15:00:16.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.873+0000 7fbc1e200640 1 -- 172.21.15.143:0/603856416 wait complete. 2024-01-16T15:00:16.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.875+0000 7fbc1e200640 1 Processor -- start 2024-01-16T15:00:16.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.875+0000 7fbc1e200640 1 -- start start 2024-01-16T15:00:16.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.876+0000 7fbc1e200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbbf8003970 0x7fbc181f7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:16.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.876+0000 7fbc1e200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc181650a0 0x7fbc1815acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:16.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.876+0000 7fbc1e200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc181fab00 0x7fbc18071c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:16.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.876+0000 7fbc1e200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc181fb9c0 con 0x7fbc181fab00 2024-01-16T15:00:16.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.876+0000 7fbc1e200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc181fb400 con 0x7fbc181650a0 2024-01-16T15:00:16.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.876+0000 7fbc1e200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc181fb6e0 con 0x7fbbf8003970 2024-01-16T15:00:16.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.876+0000 7fbc17e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc181fab00 0x7fbc18071c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:16.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.876+0000 7fbc17e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc181fab00 0x7fbc18071c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45166/0 (socket says 172.21.15.143:45166) 2024-01-16T15:00:16.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.876+0000 7fbc17e00640 1 -- 172.21.15.143:0/510212240 learned_addr learned my addr 172.21.15.143:0/510212240 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:16.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.876+0000 7fbc17400640 1 --2- 172.21.15.143:0/510212240 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbbf8003970 0x7fbc181f7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:16.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.877+0000 7fbc16a00640 1 --2- 172.21.15.143:0/510212240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc181650a0 0x7fbc1815acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:16.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.877+0000 7fbc17e00640 1 -- 172.21.15.143:0/510212240 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbbf8003970 msgr2=0x7fbc181f7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:16.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.877+0000 7fbc17e00640 1 --2- 172.21.15.143:0/510212240 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbbf8003970 0x7fbc181f7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:16.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.877+0000 7fbc17e00640 1 -- 172.21.15.143:0/510212240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc181650a0 msgr2=0x7fbc1815acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:16.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.877+0000 7fbc17e00640 1 --2- 172.21.15.143:0/510212240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc181650a0 0x7fbc1815acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:16.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.877+0000 7fbc17e00640 1 -- 172.21.15.143:0/510212240 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc181f76b0 con 0x7fbc181fab00 2024-01-16T15:00:16.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.877+0000 7fbc17e00640 1 --2- 172.21.15.143:0/510212240 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc181fab00 0x7fbc18071c50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbc180ceaa0 tx=0x7fbc1000df50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:16.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.877+0000 7fbc0f400640 1 -- 172.21.15.143:0/510212240 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1000b860 con 0x7fbc181fab00 2024-01-16T15:00:16.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.877+0000 7fbc0f400640 1 -- 172.21.15.143:0/510212240 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc1000b9c0 con 0x7fbc181fab00 2024-01-16T15:00:16.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.877+0000 7fbc1e200640 1 -- 172.21.15.143:0/510212240 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc181f7990 con 0x7fbc181fab00 2024-01-16T15:00:16.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.877+0000 7fbc0f400640 1 -- 172.21.15.143:0/510212240 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1000bd70 con 0x7fbc181fab00 2024-01-16T15:00:16.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.877+0000 7fbc1e200640 1 -- 172.21.15.143:0/510212240 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc1815b230 con 0x7fbc181fab00 2024-01-16T15:00:16.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.878+0000 7fbc1e200640 1 -- 172.21.15.143:0/510212240 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc181f7510 con 0x7fbc181fab00 2024-01-16T15:00:16.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.879+0000 7fbc0f400640 1 -- 172.21.15.143:0/510212240 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fbc10007110 con 0x7fbc181fab00 2024-01-16T15:00:16.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.880+0000 7fbc0f400640 1 --2- 172.21.15.143:0/510212240 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbbf006b2c0 0x7fbbf006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:16.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.880+0000 7fbc0f400640 1 -- 172.21.15.143:0/510212240 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc1008a060 con 0x7fbc181fab00 2024-01-16T15:00:16.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.881+0000 7fbc17400640 1 --2- 172.21.15.143:0/510212240 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbbf006b2c0 0x7fbbf006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:16.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.881+0000 7fbc17400640 1 --2- 172.21.15.143:0/510212240 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbbf006b2c0 0x7fbbf006d780 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbc0800ba00 tx=0x7fbc0803d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:16.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:16.884+0000 7fbc0f400640 1 -- 172.21.15.143:0/510212240 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbc181f7510 con 0x7fbc181fab00 2024-01-16T15:00:17.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.076+0000 7fbc1e200640 1 -- 172.21.15.143:0/510212240 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc181f7510 con 0x7fbc181fab00 2024-01-16T15:00:17.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.077+0000 7fbc0f400640 1 -- 172.21.15.143:0/510212240 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16565 (secure 0 0 0) 0x7fbc181f7510 con 0x7fbc181fab00 2024-01-16T15:00:17.315 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:00:17.315 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0","created":"2024-01-16T14:59:38.532601+0000","modified":"2024-01-16T15:00:04.543620+0000","last_up_change":"2024-01-16T14:59:50.983699+0000","last_in_change":"2024-01-16T14:59:42.056297+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:59:51.727720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T15:00:01.010472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T15:00:02.041228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f4c8891d-ab29-472a-907d-8446f20f7012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6801","nonce":1569293626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6805","nonce":1569293626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6814","nonce":1569293626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6809","nonce":1569293626}]},"public_addr":"172.21.15.143:6801/1569293626","cluster_addr":"172.21.15.143:6805/1569293626","heartbeat_back_addr":"172.21.15.143:6814/1569293626","heartbeat_front_addr":"172.21.15.143:6809/1569293626","state":["exists","up"]},{"osd":1,"uuid":"c4a2788d-8b89-4992-82b2-7f4b9debd509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6804","nonce":1276841168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6808","nonce":1276841168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6813","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6815","nonce":1276841168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6811","nonce":1276841168}]},"public_addr":"172.21.15.143:6804/1276841168","cluster_addr":"172.21.15.143:6808/1276841168","heartbeat_back_addr":"172.21.15.143:6815/1276841168","heartbeat_front_addr":"172.21.15.143:6811/1276841168","state":["exists","up"]},{"osd":2,"uuid":"64645c1d-31b2-4794-b0f2-7ccf8df30393","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6817","nonce":1217670170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6819","nonce":1217670170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6823","nonce":1217670170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6821","nonce":1217670170}]},"public_addr":"172.21.15.143:6817/1217670170","cluster_addr":"172.21.15.143:6819/1217670170","heartbeat_back_addr":"172.21.15.143:6823/1217670170","heartbeat_front_addr":"172.21.15.143:6821/1217670170","state":["exists","up"]},{"osd":3,"uuid":"73674129-a84e-4d31-8247-7d0179ae6df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6825","nonce":3414188012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6827","nonce":3414188012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6831","nonce":3414188012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6829","nonce":3414188012}]},"public_addr":"172.21.15.143:6825/3414188012","cluster_addr":"172.21.15.143:6827/3414188012","heartbeat_back_addr":"172.21.15.143:6831/3414188012","heartbeat_front_addr":"172.21.15.143:6829/3414188012","state":["exists","up"]},{"osd":4,"uuid":"cea28c9b-2f1c-4087-b415-9960789131d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6811","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6812","nonce":1070255716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6816","nonce":1070255716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6823","nonce":1070255716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6820","nonce":1070255716}]},"public_addr":"172.21.15.204:6812/1070255716","cluster_addr":"172.21.15.204:6816/1070255716","heartbeat_back_addr":"172.21.15.204:6823/1070255716","heartbeat_front_addr":"172.21.15.204:6820/1070255716","state":["exists","up"]},{"osd":5,"uuid":"33b63560-5574-4e78-983f-b398d67a2cb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6809","nonce":3605417837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6813","nonce":3605417837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6819","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6821","nonce":3605417837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6817","nonce":3605417837}]},"public_addr":"172.21.15.204:6809/3605417837","cluster_addr":"172.21.15.204:6813/3605417837","heartbeat_back_addr":"172.21.15.204:6821/3605417837","heartbeat_front_addr":"172.21.15.204:6817/3605417837","state":["exists","up"]},{"osd":6,"uuid":"df397427-8940-4eee-9c79-eece669d735d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6801","nonce":1085000534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6803","nonce":1085000534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6808","nonce":1085000534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6805","nonce":1085000534}]},"public_addr":"172.21.15.204:6801/1085000534","cluster_addr":"172.21.15.204:6803/1085000534","heartbeat_back_addr":"172.21.15.204:6808/1085000534","heartbeat_front_addr":"172.21.15.204:6805/1085000534","state":["exists","up"]},{"osd":7,"uuid":"a2e5a15c-8312-405a-ac02-f7e1fc9df2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6825","nonce":1337220609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6827","nonce":1337220609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6831","nonce":1337220609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6829","nonce":1337220609}]},"public_addr":"172.21.15.204:6825/1337220609","cluster_addr":"172.21.15.204:6827/1337220609","heartbeat_back_addr":"172.21.15.204:6831/1337220609","heartbeat_front_addr":"172.21.15.204:6829/1337220609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:49.984507+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.395306+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.632998+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.373627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.342920+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.438366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.331926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.406657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T15:00:17.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.080+0000 7fbc1e200640 1 -- 172.21.15.143:0/510212240 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbbf006b2c0 msgr2=0x7fbbf006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:17.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.080+0000 7fbc1e200640 1 --2- 172.21.15.143:0/510212240 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbbf006b2c0 0x7fbbf006d780 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbc0800ba00 tx=0x7fbc0803d040 comp rx=0 tx=0).stop 2024-01-16T15:00:17.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.080+0000 7fbc1e200640 1 -- 172.21.15.143:0/510212240 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc181fab00 msgr2=0x7fbc18071c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:17.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.080+0000 7fbc1e200640 1 --2- 172.21.15.143:0/510212240 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc181fab00 0x7fbc18071c50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbc180ceaa0 tx=0x7fbc1000df50 comp rx=0 tx=0).stop 2024-01-16T15:00:17.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.081+0000 7fbc1e200640 1 -- 172.21.15.143:0/510212240 shutdown_connections 2024-01-16T15:00:17.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.081+0000 7fbc1e200640 1 --2- 172.21.15.143:0/510212240 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbbf006b2c0 0x7fbbf006d780 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:17.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.081+0000 7fbc1e200640 1 --2- 172.21.15.143:0/510212240 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc181fab00 0x7fbc18071c50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:17.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.081+0000 7fbc1e200640 1 --2- 172.21.15.143:0/510212240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc181650a0 0x7fbc1815acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:17.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.081+0000 7fbc1e200640 1 --2- 172.21.15.143:0/510212240 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbbf8003970 0x7fbc181f7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:17.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.081+0000 7fbc1e200640 1 -- 172.21.15.143:0/510212240 >> 172.21.15.143:0/510212240 conn(0x7fbc1806a920 msgr2=0x7fbc180b3170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:17.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.081+0000 7fbc1e200640 1 -- 172.21.15.143:0/510212240 shutdown_connections 2024-01-16T15:00:17.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.081+0000 7fbc1e200640 1 -- 172.21.15.143:0/510212240 wait complete. 2024-01-16T15:00:17.317 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-16T14:59:51.727720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-16T15:00:01.010472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-16T15:00:02.041228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-16T15:00:17.317 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-16T15:00:17.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.558+0000 7f7055000640 1 Processor -- start 2024-01-16T15:00:17.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.558+0000 7f7055000640 1 -- start start 2024-01-16T15:00:17.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.560+0000 7f7055000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f705013ea00 0x7f70500cede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:17.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.560+0000 7f7055000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f705015c630 con 0x7f705013e3e0 2024-01-16T15:00:17.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.560+0000 7f7055000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f705015ba20 con 0x7f70500cf320 2024-01-16T15:00:17.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.560+0000 7f7055000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f705013e7c0 con 0x7f705013ea00 2024-01-16T15:00:17.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.560+0000 7f704de00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f705013e3e0 0x7f70500ce3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:37974/0 (socket says 172.21.15.143:37974) 2024-01-16T15:00:17.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.560+0000 7f704de00640 1 -- 172.21.15.143:0/582583002 learned_addr learned my addr 172.21.15.143:0/582583002 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:17.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.560+0000 7f7045200640 1 --2- 172.21.15.143:0/582583002 >> v2:172.21.15.143:3300/0 conn(0x7f705013ea00 0x7f70500cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:17.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.561+0000 7f704d400640 1 -- 172.21.15.143:0/582583002 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3606168542 0 0) 0x7f705015c630 con 0x7f705013e3e0 2024-01-16T15:00:17.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.561+0000 7f704d400640 1 -- 172.21.15.143:0/582583002 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7030003520 con 0x7f705013e3e0 2024-01-16T15:00:17.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.561+0000 7f7045200640 1 -- 172.21.15.143:0/582583002 >> v1:172.21.15.204:6789/0 conn(0x7f70500cf320 legacy=0x7f70500700b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:17.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.561+0000 7f7045200640 1 -- 172.21.15.143:0/582583002 >> v1:172.21.15.143:6789/0 conn(0x7f705013e3e0 legacy=0x7f70500ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:17.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.561+0000 7f7045200640 1 -- 172.21.15.143:0/582583002 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f705015bd00 con 0x7f705013ea00 2024-01-16T15:00:17.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.562+0000 7f7045200640 1 --2- 172.21.15.143:0/582583002 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f705013ea00 0x7f70500cede0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f70380097d0 tx=0x7f703802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbe9f2f7a8f843b2 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:17.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.562+0000 7f704d400640 1 -- 172.21.15.143:0/582583002 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703802d040 con 0x7f705013ea00 2024-01-16T15:00:17.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.562+0000 7f704d400640 1 -- 172.21.15.143:0/582583002 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7038031810 con 0x7f705013ea00 2024-01-16T15:00:17.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.562+0000 7f704d400640 1 -- 172.21.15.143:0/582583002 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7038031ba0 con 0x7f705013ea00 2024-01-16T15:00:17.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.563+0000 7f7055000640 1 -- 172.21.15.143:0/582583002 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f705013ea00 msgr2=0x7f70500cede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:17.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.563+0000 7f7055000640 1 --2- 172.21.15.143:0/582583002 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f705013ea00 0x7f70500cede0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f70380097d0 tx=0x7f703802b8b0 comp rx=0 tx=0).stop 2024-01-16T15:00:17.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.563+0000 7f7055000640 1 -- 172.21.15.143:0/582583002 shutdown_connections 2024-01-16T15:00:17.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.563+0000 7f7055000640 1 --2- 172.21.15.143:0/582583002 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f705013ea00 0x7f70500cede0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:17.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.563+0000 7f7055000640 1 -- 172.21.15.143:0/582583002 >> 172.21.15.143:0/582583002 conn(0x7f70500b2e40 msgr2=0x7f70500b3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:17.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.563+0000 7f7055000640 1 -- 172.21.15.143:0/582583002 shutdown_connections 2024-01-16T15:00:17.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.563+0000 7f7055000640 1 -- 172.21.15.143:0/582583002 wait complete. 2024-01-16T15:00:17.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.565+0000 7f7055000640 1 Processor -- start 2024-01-16T15:00:17.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.565+0000 7f7055000640 1 -- start start 2024-01-16T15:00:17.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.566+0000 7f7055000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70500cf320 0x7f70501f7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:17.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.566+0000 7f7055000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f705013e3e0 0x7f70500cc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:17.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.566+0000 7f7055000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f705013ea00 0x7f705014e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:17.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.566+0000 7f7055000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705006c2d0 con 0x7f705013ea00 2024-01-16T15:00:17.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.566+0000 7f7055000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70501f7580 con 0x7f70500cf320 2024-01-16T15:00:17.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.566+0000 7f7055000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70500b58b0 con 0x7f705013e3e0 2024-01-16T15:00:17.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.566+0000 7f704e800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f705013ea00 0x7f705014e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:17.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.566+0000 7f704e800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f705013ea00 0x7f705014e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45186/0 (socket says 172.21.15.143:45186) 2024-01-16T15:00:17.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.566+0000 7f704e800640 1 -- 172.21.15.143:0/758244652 learned_addr learned my addr 172.21.15.143:0/758244652 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:17.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.566+0000 7f704de00640 1 --2- 172.21.15.143:0/758244652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70500cf320 0x7f70501f7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:17.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.566+0000 7f7045200640 1 --2- 172.21.15.143:0/758244652 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f705013e3e0 0x7f70500cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:17.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.567+0000 7f704e800640 1 -- 172.21.15.143:0/758244652 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f705013e3e0 msgr2=0x7f70500cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:17.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.567+0000 7f704e800640 1 --2- 172.21.15.143:0/758244652 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f705013e3e0 0x7f70500cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:17.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.567+0000 7f704e800640 1 -- 172.21.15.143:0/758244652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70500cf320 msgr2=0x7f70501f7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:17.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.567+0000 7f704e800640 1 --2- 172.21.15.143:0/758244652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70500cf320 0x7f70501f7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:17.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.567+0000 7f704e800640 1 -- 172.21.15.143:0/758244652 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7038009400 con 0x7f705013ea00 2024-01-16T15:00:17.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.567+0000 7f7045200640 1 --2- 172.21.15.143:0/758244652 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f705013e3e0 0x7f70500cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:17.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.567+0000 7f704e800640 1 --2- 172.21.15.143:0/758244652 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f705013ea00 0x7f705014e2b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7048001eb0 tx=0x7f704800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:17.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.567+0000 7f7046a00640 1 -- 172.21.15.143:0/758244652 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70480052d0 con 0x7f705013ea00 2024-01-16T15:00:17.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.567+0000 7f7046a00640 1 -- 172.21.15.143:0/758244652 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7048005430 con 0x7f705013ea00 2024-01-16T15:00:17.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.567+0000 7f7055000640 1 -- 172.21.15.143:0/758244652 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70500b5010 con 0x7f705013ea00 2024-01-16T15:00:17.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.567+0000 7f7046a00640 1 -- 172.21.15.143:0/758244652 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7048003810 con 0x7f705013ea00 2024-01-16T15:00:17.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.567+0000 7f7055000640 1 -- 172.21.15.143:0/758244652 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70500b55d0 con 0x7f705013ea00 2024-01-16T15:00:17.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.568+0000 7f7055000640 1 -- 172.21.15.143:0/758244652 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f705006c030 con 0x7f705013ea00 2024-01-16T15:00:17.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.569+0000 7f7046a00640 1 -- 172.21.15.143:0/758244652 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f7048003970 con 0x7f705013ea00 2024-01-16T15:00:17.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.570+0000 7f7046a00640 1 --2- 172.21.15.143:0/758244652 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f702806b2c0 0x7f702806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:17.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.570+0000 7f7046a00640 1 -- 172.21.15.143:0/758244652 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7048089180 con 0x7f705013ea00 2024-01-16T15:00:17.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.570+0000 7f704de00640 1 --2- 172.21.15.143:0/758244652 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f702806b2c0 0x7f702806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:17.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.571+0000 7f704de00640 1 --2- 172.21.15.143:0/758244652 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f702806b2c0 0x7f702806d780 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7040001560 tx=0x7f7040009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:17.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.575+0000 7f7046a00640 1 -- 172.21.15.143:0/758244652 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f705006c030 con 0x7f705013ea00 2024-01-16T15:00:17.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.762+0000 7f7055000640 1 -- 172.21.15.143:0/758244652 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f705006c410 con 0x7f705013ea00 2024-01-16T15:00:17.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.763+0000 7f7046a00640 1 -- 172.21.15.143:0/758244652 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f705006c410 con 0x7f705013ea00 2024-01-16T15:00:17.745 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-16T15:00:17.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.766+0000 7f7055000640 1 -- 172.21.15.143:0/758244652 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f702806b2c0 msgr2=0x7f702806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:17.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.766+0000 7f7055000640 1 --2- 172.21.15.143:0/758244652 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f702806b2c0 0x7f702806d780 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7040001560 tx=0x7f7040009040 comp rx=0 tx=0).stop 2024-01-16T15:00:17.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.766+0000 7f7055000640 1 -- 172.21.15.143:0/758244652 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f705013ea00 msgr2=0x7f705014e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:17.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.766+0000 7f7055000640 1 --2- 172.21.15.143:0/758244652 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f705013ea00 0x7f705014e2b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7048001eb0 tx=0x7f704800c890 comp rx=0 tx=0).stop 2024-01-16T15:00:17.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.766+0000 7f7055000640 1 -- 172.21.15.143:0/758244652 shutdown_connections 2024-01-16T15:00:17.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.766+0000 7f7055000640 1 --2- 172.21.15.143:0/758244652 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f702806b2c0 0x7f702806d780 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:17.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.766+0000 7f7055000640 1 --2- 172.21.15.143:0/758244652 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f705013ea00 0x7f705014e2b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:17.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.766+0000 7f7055000640 1 --2- 172.21.15.143:0/758244652 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f705013e3e0 0x7f70500cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:17.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.766+0000 7f7055000640 1 --2- 172.21.15.143:0/758244652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70500cf320 0x7f70501f7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:17.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.766+0000 7f7055000640 1 -- 172.21.15.143:0/758244652 >> 172.21.15.143:0/758244652 conn(0x7f70500b2e40 msgr2=0x7f70500b0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:17.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.766+0000 7f7055000640 1 -- 172.21.15.143:0/758244652 shutdown_connections 2024-01-16T15:00:17.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.766+0000 7f7055000640 1 -- 172.21.15.143:0/758244652 wait complete. 2024-01-16T15:00:17.765 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-16T15:00:17.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.991+0000 7f5185600640 1 Processor -- start 2024-01-16T15:00:17.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.991+0000 7f5185600640 1 -- start start 2024-01-16T15:00:17.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.993+0000 7f5185600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f518015d7d0 0x7f51800cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:17.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.993+0000 7f5185600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f51800d1f40 0x7f51800ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:17.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.993+0000 7f5185600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51800d15e0 con 0x7f518015d3f0 2024-01-16T15:00:17.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.993+0000 7f5185600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51800d0280 con 0x7f51800d1f40 2024-01-16T15:00:17.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.993+0000 7f5185600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51800b0eb0 con 0x7f518015d7d0 2024-01-16T15:00:17.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.993+0000 7f5184c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f51800d1f40 0x7f51800ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:17.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.993+0000 7f5184c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f51800d1f40 0x7f51800ce280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45192/0 (socket says 172.21.15.143:45192) 2024-01-16T15:00:17.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.993+0000 7f5184c00640 1 -- 172.21.15.143:0/3540840029 learned_addr learned my addr 172.21.15.143:0/3540840029 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:17.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.994+0000 7f5177e00640 1 --2- 172.21.15.143:0/3540840029 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f518015d7d0 0x7f51800cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:17.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.994+0000 7f5184c00640 1 -- 172.21.15.143:0/3540840029 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f518015d7d0 msgr2=0x7f51800cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:17.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.994+0000 7f5184c00640 1 --2- 172.21.15.143:0/3540840029 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f518015d7d0 0x7f51800cdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:17.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.994+0000 7f5184c00640 1 -- 172.21.15.143:0/3540840029 >> v1:172.21.15.204:6789/0 conn(0x7f518015d3f0 legacy=0x7f51800d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:17.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.994+0000 7f5184c00640 1 -- 172.21.15.143:0/3540840029 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51800d1300 con 0x7f51800d1f40 2024-01-16T15:00:17.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.995+0000 7f5184c00640 1 --2- 172.21.15.143:0/3540840029 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51800d1f40 0x7f51800ce280 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5178009e50 tx=0x7f517802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c52b9ea0d0cb56a1 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:17.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.995+0000 7f517f400640 1 -- 172.21.15.143:0/3540840029 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517802ca70 con 0x7f51800d1f40 2024-01-16T15:00:17.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.995+0000 7f517f400640 1 -- 172.21.15.143:0/3540840029 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517802cbd0 con 0x7f51800d1f40 2024-01-16T15:00:17.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.995+0000 7f517f400640 1 -- 172.21.15.143:0/3540840029 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51780343f0 con 0x7f51800d1f40 2024-01-16T15:00:17.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.996+0000 7f5185600640 1 -- 172.21.15.143:0/3540840029 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51800d1f40 msgr2=0x7f51800ce280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:17.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.996+0000 7f5185600640 1 --2- 172.21.15.143:0/3540840029 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51800d1f40 0x7f51800ce280 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5178009e50 tx=0x7f517802bed0 comp rx=0 tx=0).stop 2024-01-16T15:00:17.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.996+0000 7f5185600640 1 -- 172.21.15.143:0/3540840029 shutdown_connections 2024-01-16T15:00:17.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.996+0000 7f5185600640 1 --2- 172.21.15.143:0/3540840029 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51800d1f40 0x7f51800ce280 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:17.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.996+0000 7f5185600640 1 --2- 172.21.15.143:0/3540840029 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f518015d7d0 0x7f51800cdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:17.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.996+0000 7f5185600640 1 -- 172.21.15.143:0/3540840029 >> 172.21.15.143:0/3540840029 conn(0x7f518009c5b0 msgr2=0x7f51800d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:17.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.996+0000 7f5185600640 1 -- 172.21.15.143:0/3540840029 shutdown_connections 2024-01-16T15:00:17.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.996+0000 7f5185600640 1 -- 172.21.15.143:0/3540840029 wait complete. 2024-01-16T15:00:17.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.998+0000 7f5185600640 1 Processor -- start 2024-01-16T15:00:17.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.998+0000 7f5185600640 1 -- start start 2024-01-16T15:00:17.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.999+0000 7f5185600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51800d1f40 0x7f518015ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:17.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.999+0000 7f5185600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f518015d3f0 0x7f51800c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:17.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.999+0000 7f5185600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f518015d7d0 0x7f51800b8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:17.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.999+0000 7f5185600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51800c8ca0 con 0x7f518015d7d0 2024-01-16T15:00:17.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.999+0000 7f5185600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51800c8f80 con 0x7f51800d1f40 2024-01-16T15:00:17.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.999+0000 7f5185600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51800c9260 con 0x7f518015d3f0 2024-01-16T15:00:17.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.999+0000 7f5184c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f518015d7d0 0x7f51800b8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:17.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.999+0000 7f5184c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f518015d7d0 0x7f51800b8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45196/0 (socket says 172.21.15.143:45196) 2024-01-16T15:00:17.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.999+0000 7f5184c00640 1 -- 172.21.15.143:0/1855276934 learned_addr learned my addr 172.21.15.143:0/1855276934 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:17.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.999+0000 7f5177e00640 1 --2- 172.21.15.143:0/1855276934 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f518015d3f0 0x7f51800c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:17.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.999+0000 7f5184c00640 1 -- 172.21.15.143:0/1855276934 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f518015d3f0 msgr2=0x7f51800c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:17.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.999+0000 7f5184c00640 1 --2- 172.21.15.143:0/1855276934 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f518015d3f0 0x7f51800c82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:17.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.999+0000 7f5184c00640 1 -- 172.21.15.143:0/1855276934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51800d1f40 msgr2=0x7f518015ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:17.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.999+0000 7f5184c00640 1 --2- 172.21.15.143:0/1855276934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51800d1f40 0x7f518015ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:17.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:17.999+0000 7f5184c00640 1 -- 172.21.15.143:0/1855276934 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5178009b30 con 0x7f518015d7d0 2024-01-16T15:00:17.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.000+0000 7f5184c00640 1 --2- 172.21.15.143:0/1855276934 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f518015d7d0 0x7f51800b8060 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5178036040 tx=0x7f5178004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:17.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.000+0000 7f517cc00640 1 -- 172.21.15.143:0/1855276934 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5178009510 con 0x7f518015d7d0 2024-01-16T15:00:17.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.000+0000 7f517cc00640 1 -- 172.21.15.143:0/1855276934 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517803e450 con 0x7f518015d7d0 2024-01-16T15:00:17.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.000+0000 7f517cc00640 1 -- 172.21.15.143:0/1855276934 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517803e7e0 con 0x7f518015d7d0 2024-01-16T15:00:17.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.000+0000 7f5185600640 1 -- 172.21.15.143:0/1855276934 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51800c9540 con 0x7f518015d7d0 2024-01-16T15:00:17.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.000+0000 7f5185600640 1 -- 172.21.15.143:0/1855276934 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f518015b2e0 con 0x7f518015d7d0 2024-01-16T15:00:17.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.001+0000 7f5185600640 1 -- 172.21.15.143:0/1855276934 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f518004e840 con 0x7f518015d7d0 2024-01-16T15:00:17.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.002+0000 7f517cc00640 1 -- 172.21.15.143:0/1855276934 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f517803e940 con 0x7f518015d7d0 2024-01-16T15:00:17.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.003+0000 7f517cc00640 1 --2- 172.21.15.143:0/1855276934 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f515806b2c0 0x7f515806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:17.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.003+0000 7f517cc00640 1 -- 172.21.15.143:0/1855276934 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f51780ac9a0 con 0x7f518015d7d0 2024-01-16T15:00:17.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.003+0000 7f517fe00640 1 --2- 172.21.15.143:0/1855276934 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f515806b2c0 0x7f515806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:17.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.004+0000 7f517fe00640 1 --2- 172.21.15.143:0/1855276934 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f515806b2c0 0x7f515806d780 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f51800b8ca0 tx=0x7f51700079b0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:17.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.008+0000 7f517cc00640 1 -- 172.21.15.143:0/1855276934 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5178077930 con 0x7f518015d7d0 2024-01-16T15:00:18.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.198+0000 7f5185600640 1 -- 172.21.15.143:0/1855276934 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f51800d06f0 con 0x7f518015d7d0 2024-01-16T15:00:18.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.199+0000 7f517cc00640 1 -- 172.21.15.143:0/1855276934 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f51800d06f0 con 0x7f518015d7d0 2024-01-16T15:00:18.180 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2024-01-16T15:00:18.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.202+0000 7f5185600640 1 -- 172.21.15.143:0/1855276934 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f515806b2c0 msgr2=0x7f515806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:18.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.202+0000 7f5185600640 1 --2- 172.21.15.143:0/1855276934 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f515806b2c0 0x7f515806d780 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f51800b8ca0 tx=0x7f51700079b0 comp rx=0 tx=0).stop 2024-01-16T15:00:18.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.202+0000 7f5185600640 1 -- 172.21.15.143:0/1855276934 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f518015d7d0 msgr2=0x7f51800b8060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:18.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.202+0000 7f5185600640 1 --2- 172.21.15.143:0/1855276934 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f518015d7d0 0x7f51800b8060 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5178036040 tx=0x7f5178004a40 comp rx=0 tx=0).stop 2024-01-16T15:00:18.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.202+0000 7f5185600640 1 -- 172.21.15.143:0/1855276934 shutdown_connections 2024-01-16T15:00:18.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.202+0000 7f5185600640 1 --2- 172.21.15.143:0/1855276934 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f515806b2c0 0x7f515806d780 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:18.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.202+0000 7f5185600640 1 --2- 172.21.15.143:0/1855276934 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f518015d7d0 0x7f51800b8060 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:18.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.202+0000 7f5185600640 1 --2- 172.21.15.143:0/1855276934 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f518015d3f0 0x7f51800c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:18.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.202+0000 7f5185600640 1 --2- 172.21.15.143:0/1855276934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51800d1f40 0x7f518015ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:18.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.202+0000 7f5185600640 1 -- 172.21.15.143:0/1855276934 >> 172.21.15.143:0/1855276934 conn(0x7f518009c5b0 msgr2=0x7f518015c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:18.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.202+0000 7f5185600640 1 -- 172.21.15.143:0/1855276934 shutdown_connections 2024-01-16T15:00:18.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.202+0000 7f5185600640 1 -- 172.21.15.143:0/1855276934 wait complete. 2024-01-16T15:00:18.201 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-16T15:00:18.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.426+0000 7fb1cb000640 1 Processor -- start 2024-01-16T15:00:18.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.426+0000 7fb1cb000640 1 -- start start 2024-01-16T15:00:18.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.427+0000 7fb1cb000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb1c406de10 0x7fb1c415e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:18.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.427+0000 7fb1cb000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb1c40c49f0 0x7fb1c40705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:18.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.427+0000 7fb1cb000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1c415ec90 con 0x7fb1c4072160 2024-01-16T15:00:18.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.427+0000 7fb1cb000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1c40cd530 con 0x7fb1c406de10 2024-01-16T15:00:18.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.427+0000 7fb1cb000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1c40c4ed0 con 0x7fb1c40c49f0 2024-01-16T15:00:18.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.428+0000 7fb1bb200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb1c406de10 0x7fb1c415e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:18.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.428+0000 7fb1bb200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb1c406de10 0x7fb1c415e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45204/0 (socket says 172.21.15.143:45204) 2024-01-16T15:00:18.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.428+0000 7fb1bb200640 1 -- 172.21.15.143:0/306849026 learned_addr learned my addr 172.21.15.143:0/306849026 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:18.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.428+0000 7fb1c9000640 1 --2- 172.21.15.143:0/306849026 >> v2:172.21.15.204:3300/0 conn(0x7fb1c40c49f0 0x7fb1c40705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:18.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.428+0000 7fb1bb200640 1 -- 172.21.15.143:0/306849026 >> v2:172.21.15.204:3300/0 conn(0x7fb1c40c49f0 msgr2=0x7fb1c40705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:18.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.428+0000 7fb1bb200640 1 --2- 172.21.15.143:0/306849026 >> v2:172.21.15.204:3300/0 conn(0x7fb1c40c49f0 0x7fb1c40705c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:18.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.428+0000 7fb1bb200640 1 -- 172.21.15.143:0/306849026 >> v1:172.21.15.204:6789/0 conn(0x7fb1c4072160 legacy=0x7fb1c406d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:18.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.428+0000 7fb1bb200640 1 -- 172.21.15.143:0/306849026 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c40c51b0 con 0x7fb1c406de10 2024-01-16T15:00:18.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.429+0000 7fb1bb200640 1 --2- 172.21.15.143:0/306849026 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1c406de10 0x7fb1c415e750 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4072a30 tx=0x7fb1ac02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=444dc7698b87eda8 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:18.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.429+0000 7fb1c3400640 1 -- 172.21.15.143:0/306849026 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac02ca80 con 0x7fb1c406de10 2024-01-16T15:00:18.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.429+0000 7fb1c3400640 1 -- 172.21.15.143:0/306849026 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1ac02cbe0 con 0x7fb1c406de10 2024-01-16T15:00:18.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.430+0000 7fb1c3400640 1 -- 172.21.15.143:0/306849026 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac0343f0 con 0x7fb1c406de10 2024-01-16T15:00:18.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.430+0000 7fb1cb000640 1 -- 172.21.15.143:0/306849026 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1c406de10 msgr2=0x7fb1c415e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:18.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.430+0000 7fb1cb000640 1 --2- 172.21.15.143:0/306849026 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1c406de10 0x7fb1c415e750 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4072a30 tx=0x7fb1ac02bee0 comp rx=0 tx=0).stop 2024-01-16T15:00:18.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.430+0000 7fb1cb000640 1 -- 172.21.15.143:0/306849026 shutdown_connections 2024-01-16T15:00:18.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.430+0000 7fb1cb000640 1 --2- 172.21.15.143:0/306849026 >> v2:172.21.15.204:3300/0 conn(0x7fb1c40c49f0 0x7fb1c40705c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:18.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.430+0000 7fb1cb000640 1 --2- 172.21.15.143:0/306849026 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1c406de10 0x7fb1c415e750 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:18.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.430+0000 7fb1cb000640 1 -- 172.21.15.143:0/306849026 >> 172.21.15.143:0/306849026 conn(0x7fb1c40b0c50 msgr2=0x7fb1c40b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:18.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.430+0000 7fb1cb000640 1 -- 172.21.15.143:0/306849026 shutdown_connections 2024-01-16T15:00:18.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.430+0000 7fb1cb000640 1 -- 172.21.15.143:0/306849026 wait complete. 2024-01-16T15:00:18.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.432+0000 7fb1cb000640 1 Processor -- start 2024-01-16T15:00:18.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.432+0000 7fb1cb000640 1 -- start start 2024-01-16T15:00:18.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.433+0000 7fb1cb000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1c406de10 0x7fb1c415a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:18.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.433+0000 7fb1cb000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1c4072160 0x7fb1c414dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:18.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.433+0000 7fb1cb000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1c40c49f0 0x7fb1c4153130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:18.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.433+0000 7fb1cb000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c40ca590 con 0x7fb1c406de10 2024-01-16T15:00:18.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.433+0000 7fb1cb000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c4159790 con 0x7fb1c4072160 2024-01-16T15:00:18.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.433+0000 7fb1cb000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c4159a70 con 0x7fb1c40c49f0 2024-01-16T15:00:18.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.433+0000 7fb1c3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1c406de10 0x7fb1c415a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:18.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.433+0000 7fb1c3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1c406de10 0x7fb1c415a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45216/0 (socket says 172.21.15.143:45216) 2024-01-16T15:00:18.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.433+0000 7fb1c3e00640 1 -- 172.21.15.143:0/715311647 learned_addr learned my addr 172.21.15.143:0/715311647 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:18.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.434+0000 7fb1c9000640 1 --2- 172.21.15.143:0/715311647 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1c40c49f0 0x7fb1c4153130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:18.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.434+0000 7fb1c3e00640 1 -- 172.21.15.143:0/715311647 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1c40c49f0 msgr2=0x7fb1c4153130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:18.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.434+0000 7fb1c3e00640 1 --2- 172.21.15.143:0/715311647 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1c40c49f0 0x7fb1c4153130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:18.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.434+0000 7fb1c3e00640 1 -- 172.21.15.143:0/715311647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1c4072160 msgr2=0x7fb1c414dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:18.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.434+0000 7fb1c3e00640 1 --2- 172.21.15.143:0/715311647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1c4072160 0x7fb1c414dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:18.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.434+0000 7fb1c3e00640 1 -- 172.21.15.143:0/715311647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1ac009b30 con 0x7fb1c406de10 2024-01-16T15:00:18.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.434+0000 7fb1c3e00640 1 --2- 172.21.15.143:0/715311647 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1c406de10 0x7fb1c415a470 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb1b400c410 tx=0x7fb1b400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:18.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.434+0000 7fb1c0c00640 1 -- 172.21.15.143:0/715311647 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b40035f0 con 0x7fb1c406de10 2024-01-16T15:00:18.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.434+0000 7fb1c0c00640 1 -- 172.21.15.143:0/715311647 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b4003750 con 0x7fb1c406de10 2024-01-16T15:00:18.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.434+0000 7fb1c0c00640 1 -- 172.21.15.143:0/715311647 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4003b40 con 0x7fb1c406de10 2024-01-16T15:00:18.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.434+0000 7fb1cb000640 1 -- 172.21.15.143:0/715311647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c4159d50 con 0x7fb1c406de10 2024-01-16T15:00:18.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.435+0000 7fb1cb000640 1 -- 172.21.15.143:0/715311647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c414e140 con 0x7fb1c406de10 2024-01-16T15:00:18.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.436+0000 7fb1cb000640 1 -- 172.21.15.143:0/715311647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c404e8b0 con 0x7fb1c406de10 2024-01-16T15:00:18.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.437+0000 7fb1c0c00640 1 -- 172.21.15.143:0/715311647 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fb1b4003ca0 con 0x7fb1c406de10 2024-01-16T15:00:18.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.437+0000 7fb1c0c00640 1 --2- 172.21.15.143:0/715311647 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb19c06b2f0 0x7fb19c06d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:18.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.438+0000 7fb1c0c00640 1 -- 172.21.15.143:0/715311647 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb1b4088d00 con 0x7fb1c406de10 2024-01-16T15:00:18.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.438+0000 7fb1bb200640 1 --2- 172.21.15.143:0/715311647 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb19c06b2f0 0x7fb19c06d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:18.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.438+0000 7fb1bb200640 1 --2- 172.21.15.143:0/715311647 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb19c06b2f0 0x7fb19c06d7b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac035000 tx=0x7fb1ac02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:18.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.442+0000 7fb1c0c00640 1 -- 172.21.15.143:0/715311647 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb1b4053c00 con 0x7fb1c406de10 2024-01-16T15:00:18.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.636+0000 7fb1cb000640 1 -- 172.21.15.143:0/715311647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb1c415dab0 con 0x7fb1c406de10 2024-01-16T15:00:18.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.637+0000 7fb1c0c00640 1 -- 172.21.15.143:0/715311647 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb1b4057220 con 0x7fb1c406de10 2024-01-16T15:00:18.618 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2024-01-16T15:00:18.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.639+0000 7fb1cb000640 1 -- 172.21.15.143:0/715311647 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb19c06b2f0 msgr2=0x7fb19c06d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:18.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.639+0000 7fb1cb000640 1 --2- 172.21.15.143:0/715311647 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb19c06b2f0 0x7fb19c06d7b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac035000 tx=0x7fb1ac02e000 comp rx=0 tx=0).stop 2024-01-16T15:00:18.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.639+0000 7fb1cb000640 1 -- 172.21.15.143:0/715311647 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1c406de10 msgr2=0x7fb1c415a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:18.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.639+0000 7fb1cb000640 1 --2- 172.21.15.143:0/715311647 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1c406de10 0x7fb1c415a470 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb1b400c410 tx=0x7fb1b400c8d0 comp rx=0 tx=0).stop 2024-01-16T15:00:18.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.640+0000 7fb1cb000640 1 -- 172.21.15.143:0/715311647 shutdown_connections 2024-01-16T15:00:18.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.640+0000 7fb1cb000640 1 --2- 172.21.15.143:0/715311647 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb19c06b2f0 0x7fb19c06d7b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:18.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.640+0000 7fb1cb000640 1 --2- 172.21.15.143:0/715311647 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1c40c49f0 0x7fb1c4153130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:18.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.640+0000 7fb1cb000640 1 --2- 172.21.15.143:0/715311647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1c4072160 0x7fb1c414dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:18.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.640+0000 7fb1cb000640 1 --2- 172.21.15.143:0/715311647 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1c406de10 0x7fb1c415a470 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:18.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.640+0000 7fb1cb000640 1 -- 172.21.15.143:0/715311647 >> 172.21.15.143:0/715311647 conn(0x7fb1c40b0c50 msgr2=0x7fb1c40b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:18.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.640+0000 7fb1cb000640 1 -- 172.21.15.143:0/715311647 shutdown_connections 2024-01-16T15:00:18.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.640+0000 7fb1cb000640 1 -- 172.21.15.143:0/715311647 wait complete. 2024-01-16T15:00:18.638 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-16T15:00:18.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.858+0000 7f0d49800640 1 Processor -- start 2024-01-16T15:00:18.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.859+0000 7f0d49800640 1 -- start start 2024-01-16T15:00:18.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.860+0000 7f0d49800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0d4415c7f0 0x7f0d4406db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:18.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.860+0000 7f0d49800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0d440cf450 0x7f0d440cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:18.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.860+0000 7f0d49800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0d440cdfb0 0x7f0d440ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:18.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.860+0000 7f0d49800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d44156010 con 0x7f0d4415c7f0 2024-01-16T15:00:18.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.860+0000 7f0d49800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d4406ce30 con 0x7f0d440cdfb0 2024-01-16T15:00:18.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.860+0000 7f0d49800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d440cfe10 con 0x7f0d440cf450 2024-01-16T15:00:18.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.860+0000 7f0d42a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0d4415c7f0 0x7f0d4406db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:18.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.860+0000 7f0d42a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0d4415c7f0 0x7f0d4406db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45226/0 (socket says 172.21.15.143:45226) 2024-01-16T15:00:18.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.860+0000 7f0d42a00640 1 -- 172.21.15.143:0/1312516421 learned_addr learned my addr 172.21.15.143:0/1312516421 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:18.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.861+0000 7f0d42a00640 1 -- 172.21.15.143:0/1312516421 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0d440cf450 msgr2=0x7f0d440cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:18.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.861+0000 7f0d42a00640 1 --2- 172.21.15.143:0/1312516421 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0d440cf450 0x7f0d440cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:18.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.861+0000 7f0d42a00640 1 -- 172.21.15.143:0/1312516421 >> v2:172.21.15.204:3300/0 conn(0x7f0d440cdfb0 msgr2=0x7f0d440ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:18.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.861+0000 7f0d42a00640 1 --2- 172.21.15.143:0/1312516421 >> v2:172.21.15.204:3300/0 conn(0x7f0d440cdfb0 0x7f0d440ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:18.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.861+0000 7f0d42a00640 1 -- 172.21.15.143:0/1312516421 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d440cece0 con 0x7f0d4415c7f0 2024-01-16T15:00:18.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.862+0000 7f0d42a00640 1 --2- 172.21.15.143:0/1312516421 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d4415c7f0 0x7f0d4406db20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0d440aebc0 tx=0x7f0d3402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91af5ff1daccf2ef server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:18.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.862+0000 7f0d42000640 1 -- 172.21.15.143:0/1312516421 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d34031870 con 0x7f0d4415c7f0 2024-01-16T15:00:18.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.862+0000 7f0d42000640 1 -- 172.21.15.143:0/1312516421 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d340319d0 con 0x7f0d4415c7f0 2024-01-16T15:00:18.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.862+0000 7f0d42000640 1 -- 172.21.15.143:0/1312516421 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d34031d60 con 0x7f0d4415c7f0 2024-01-16T15:00:18.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.863+0000 7f0d49800640 1 -- 172.21.15.143:0/1312516421 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d4415c7f0 msgr2=0x7f0d4406db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:18.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.863+0000 7f0d49800640 1 --2- 172.21.15.143:0/1312516421 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d4415c7f0 0x7f0d4406db20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0d440aebc0 tx=0x7f0d3402b910 comp rx=0 tx=0).stop 2024-01-16T15:00:18.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.863+0000 7f0d49800640 1 -- 172.21.15.143:0/1312516421 shutdown_connections 2024-01-16T15:00:18.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.863+0000 7f0d49800640 1 --2- 172.21.15.143:0/1312516421 >> v2:172.21.15.204:3300/0 conn(0x7f0d440cdfb0 0x7f0d440ce440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:18.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.863+0000 7f0d49800640 1 --2- 172.21.15.143:0/1312516421 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0d440cf450 0x7f0d440cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:18.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.863+0000 7f0d49800640 1 --2- 172.21.15.143:0/1312516421 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d4415c7f0 0x7f0d4406db20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:18.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.863+0000 7f0d49800640 1 -- 172.21.15.143:0/1312516421 >> 172.21.15.143:0/1312516421 conn(0x7f0d4406a920 msgr2=0x7f0d4406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:18.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.863+0000 7f0d49800640 1 -- 172.21.15.143:0/1312516421 shutdown_connections 2024-01-16T15:00:18.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.863+0000 7f0d49800640 1 -- 172.21.15.143:0/1312516421 wait complete. 2024-01-16T15:00:18.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.865+0000 7f0d49800640 1 Processor -- start 2024-01-16T15:00:18.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.865+0000 7f0d49800640 1 -- start start 2024-01-16T15:00:18.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.866+0000 7f0d49800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d440cdfb0 0x7f0d440ca330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:18.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.866+0000 7f0d49800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d440cf450 0x7f0d441f3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:18.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.866+0000 7f0d49800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0d4415c7f0 0x7f0d4406db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:18.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.866+0000 7f0d49800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d440c8600 con 0x7f0d440cf450 2024-01-16T15:00:18.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.866+0000 7f0d49800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d441f4920 con 0x7f0d440cdfb0 2024-01-16T15:00:18.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.866+0000 7f0d49800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d441f4080 con 0x7f0d4415c7f0 2024-01-16T15:00:18.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.866+0000 7f0d39e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d440cf450 0x7f0d441f3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:18.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.866+0000 7f0d39e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d440cf450 0x7f0d441f3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45234/0 (socket says 172.21.15.143:45234) 2024-01-16T15:00:18.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.866+0000 7f0d39e00640 1 -- 172.21.15.143:0/3478759559 learned_addr learned my addr 172.21.15.143:0/3478759559 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:18.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.866+0000 7f0d42a00640 1 --2- 172.21.15.143:0/3478759559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d440cdfb0 0x7f0d440ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:18.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.867+0000 7f0d43400640 1 --2- 172.21.15.143:0/3478759559 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0d4415c7f0 0x7f0d4406db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:18.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.867+0000 7f0d39e00640 1 -- 172.21.15.143:0/3478759559 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0d4415c7f0 msgr2=0x7f0d4406db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:18.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.867+0000 7f0d39e00640 1 --2- 172.21.15.143:0/3478759559 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0d4415c7f0 0x7f0d4406db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:18.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.867+0000 7f0d39e00640 1 -- 172.21.15.143:0/3478759559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d440cdfb0 msgr2=0x7f0d440ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:18.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.867+0000 7f0d39e00640 1 --2- 172.21.15.143:0/3478759559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d440cdfb0 0x7f0d440ca330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:18.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.867+0000 7f0d39e00640 1 -- 172.21.15.143:0/3478759559 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d340094d0 con 0x7f0d440cf450 2024-01-16T15:00:18.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.867+0000 7f0d43400640 1 --2- 172.21.15.143:0/3478759559 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0d4415c7f0 0x7f0d4406db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:18.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.867+0000 7f0d39e00640 1 --2- 172.21.15.143:0/3478759559 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d440cf450 0x7f0d441f3690 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00b4a0 tx=0x7f0d2c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:18.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.867+0000 7f0d3b400640 1 -- 172.21.15.143:0/3478759559 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c010610 con 0x7f0d440cf450 2024-01-16T15:00:18.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.867+0000 7f0d3b400640 1 -- 172.21.15.143:0/3478759559 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d2c010770 con 0x7f0d440cf450 2024-01-16T15:00:18.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.867+0000 7f0d3b400640 1 -- 172.21.15.143:0/3478759559 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c010b20 con 0x7f0d440cf450 2024-01-16T15:00:18.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.867+0000 7f0d49800640 1 -- 172.21.15.143:0/3478759559 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d441f4360 con 0x7f0d440cf450 2024-01-16T15:00:18.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.867+0000 7f0d49800640 1 -- 172.21.15.143:0/3478759559 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d441f3bd0 con 0x7f0d440cf450 2024-01-16T15:00:18.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.868+0000 7f0d49800640 1 -- 172.21.15.143:0/3478759559 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d44155b30 con 0x7f0d440cf450 2024-01-16T15:00:18.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.869+0000 7f0d3b400640 1 -- 172.21.15.143:0/3478759559 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f0d2c010c80 con 0x7f0d440cf450 2024-01-16T15:00:18.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.870+0000 7f0d3b400640 1 --2- 172.21.15.143:0/3478759559 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0d1c06b2c0 0x7f0d1c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:18.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.870+0000 7f0d3b400640 1 -- 172.21.15.143:0/3478759559 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0d2c089760 con 0x7f0d440cf450 2024-01-16T15:00:18.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.870+0000 7f0d42a00640 1 --2- 172.21.15.143:0/3478759559 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0d1c06b2c0 0x7f0d1c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:18.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.871+0000 7f0d42a00640 1 --2- 172.21.15.143:0/3478759559 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0d1c06b2c0 0x7f0d1c06d780 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0d34036000 tx=0x7f0d3403c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:18.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:18.875+0000 7f0d3b400640 1 -- 172.21.15.143:0/3478759559 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0d2c054660 con 0x7f0d440cf450 2024-01-16T15:00:19.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.092+0000 7f0d49800640 1 -- 172.21.15.143:0/3478759559 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d440c8360 con 0x7f0d440cf450 2024-01-16T15:00:19.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.094+0000 7f0d3b400640 1 -- 172.21.15.143:0/3478759559 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4916 (secure 0 0 0) 0x7f0d440c8360 con 0x7f0d440cf450 2024-01-16T15:00:19.075 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:00:19.075 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3599169408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3599169408},{"type":"v1","addr":"172.21.15.204:6834","nonce":3599169408}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/859082411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":859082411},{"type":"v1","addr":"172.21.15.204:6835","nonce":859082411}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6836/1564738611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1564738611},{"type":"v1","addr":"172.21.15.143:6836","nonce":1564738611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-16T15:00:03.050082+0000","modified":"2024-01-16T15:00:06.109664+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/3120706283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6835","nonce":3120706283},{"type":"v1","addr":"172.21.15.143:6837","nonce":3120706283}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-16T15:00:19.075 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 8 2024-01-16T15:00:19.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.096+0000 7f0d49800640 1 -- 172.21.15.143:0/3478759559 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0d1c06b2c0 msgr2=0x7f0d1c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:19.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.096+0000 7f0d49800640 1 --2- 172.21.15.143:0/3478759559 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0d1c06b2c0 0x7f0d1c06d780 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0d34036000 tx=0x7f0d3403c040 comp rx=0 tx=0).stop 2024-01-16T15:00:19.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.096+0000 7f0d49800640 1 -- 172.21.15.143:0/3478759559 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d440cf450 msgr2=0x7f0d441f3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:19.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.096+0000 7f0d49800640 1 --2- 172.21.15.143:0/3478759559 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d440cf450 0x7f0d441f3690 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00b4a0 tx=0x7f0d2c00b960 comp rx=0 tx=0).stop 2024-01-16T15:00:19.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.096+0000 7f0d49800640 1 -- 172.21.15.143:0/3478759559 shutdown_connections 2024-01-16T15:00:19.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.096+0000 7f0d49800640 1 --2- 172.21.15.143:0/3478759559 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0d1c06b2c0 0x7f0d1c06d780 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:19.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.096+0000 7f0d49800640 1 --2- 172.21.15.143:0/3478759559 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0d4415c7f0 0x7f0d4406db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:19.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.096+0000 7f0d49800640 1 --2- 172.21.15.143:0/3478759559 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d440cf450 0x7f0d441f3690 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:19.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.096+0000 7f0d49800640 1 --2- 172.21.15.143:0/3478759559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d440cdfb0 0x7f0d440ca330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:19.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.096+0000 7f0d49800640 1 -- 172.21.15.143:0/3478759559 >> 172.21.15.143:0/3478759559 conn(0x7f0d4406a920 msgr2=0x7f0d4415eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:19.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.096+0000 7f0d49800640 1 -- 172.21.15.143:0/3478759559 shutdown_connections 2024-01-16T15:00:19.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.097+0000 7f0d49800640 1 -- 172.21.15.143:0/3478759559 wait complete. 2024-01-16T15:00:19.094 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T15:00:19.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.312+0000 7f7021000640 1 Processor -- start 2024-01-16T15:00:19.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.312+0000 7f7021000640 1 -- start start 2024-01-16T15:00:19.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.314+0000 7f7021000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f701c144b30 0x7f701c15f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:19.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.314+0000 7f7021000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f701c0b1de0 0x7f701c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:19.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.314+0000 7f7021000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f701c15bf80 con 0x7f701c15c410 2024-01-16T15:00:19.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.314+0000 7f7021000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f701c0b27b0 con 0x7f701c144b30 2024-01-16T15:00:19.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.314+0000 7f7021000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f701c0b1560 con 0x7f701c0b1de0 2024-01-16T15:00:19.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.314+0000 7f701b400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f701c144b30 0x7f701c15f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:19.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.314+0000 7f701b400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f701c144b30 0x7f701c15f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45248/0 (socket says 172.21.15.143:45248) 2024-01-16T15:00:19.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.314+0000 7f701b400640 1 -- 172.21.15.143:0/1308573700 learned_addr learned my addr 172.21.15.143:0/1308573700 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:19.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.315+0000 7f701be00640 1 --2- 172.21.15.143:0/1308573700 >> v2:172.21.15.204:3300/0 conn(0x7f701c0b1de0 0x7f701c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:19.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.315+0000 7f701b400640 1 -- 172.21.15.143:0/1308573700 >> v2:172.21.15.204:3300/0 conn(0x7f701c0b1de0 msgr2=0x7f701c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:19.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.315+0000 7f701b400640 1 --2- 172.21.15.143:0/1308573700 >> v2:172.21.15.204:3300/0 conn(0x7f701c0b1de0 0x7f701c0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:19.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.315+0000 7f701b400640 1 -- 172.21.15.143:0/1308573700 >> v1:172.21.15.204:6789/0 conn(0x7f701c15c410 legacy=0x7f701c0b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:19.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.315+0000 7f701b400640 1 -- 172.21.15.143:0/1308573700 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f701c0b1280 con 0x7f701c144b30 2024-01-16T15:00:19.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.316+0000 7f701b400640 1 --2- 172.21.15.143:0/1308573700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f701c144b30 0x7f701c15f920 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f701c072a50 tx=0x7f700c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c6d454ee20e371e server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:19.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.317+0000 7f701aa00640 1 -- 172.21.15.143:0/1308573700 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700c030860 con 0x7f701c144b30 2024-01-16T15:00:19.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.317+0000 7f701aa00640 1 -- 172.21.15.143:0/1308573700 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f700c0309c0 con 0x7f701c144b30 2024-01-16T15:00:19.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.317+0000 7f701aa00640 1 -- 172.21.15.143:0/1308573700 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700c030d50 con 0x7f701c144b30 2024-01-16T15:00:19.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.317+0000 7f7021000640 1 -- 172.21.15.143:0/1308573700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f701c144b30 msgr2=0x7f701c15f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:19.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.317+0000 7f7021000640 1 --2- 172.21.15.143:0/1308573700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f701c144b30 0x7f701c15f920 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f701c072a50 tx=0x7f700c02b9a0 comp rx=0 tx=0).stop 2024-01-16T15:00:19.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.317+0000 7f7021000640 1 -- 172.21.15.143:0/1308573700 shutdown_connections 2024-01-16T15:00:19.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.317+0000 7f7021000640 1 --2- 172.21.15.143:0/1308573700 >> v2:172.21.15.204:3300/0 conn(0x7f701c0b1de0 0x7f701c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:19.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.317+0000 7f7021000640 1 --2- 172.21.15.143:0/1308573700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f701c144b30 0x7f701c15f920 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:19.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.317+0000 7f7021000640 1 -- 172.21.15.143:0/1308573700 >> 172.21.15.143:0/1308573700 conn(0x7f701c0cb830 msgr2=0x7f701c0cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:19.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.318+0000 7f7021000640 1 -- 172.21.15.143:0/1308573700 shutdown_connections 2024-01-16T15:00:19.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.318+0000 7f7021000640 1 -- 172.21.15.143:0/1308573700 wait complete. 2024-01-16T15:00:19.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.319+0000 7f7021000640 1 Processor -- start 2024-01-16T15:00:19.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.319+0000 7f7021000640 1 -- start start 2024-01-16T15:00:19.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.320+0000 7f7021000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f701c0b1de0 0x7f701c15d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:19.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.320+0000 7f7021000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f701c144b30 0x7f701c1f1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:19.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.320+0000 7f7021000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f701c15c410 0x7f701c1403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:19.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.320+0000 7f7021000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c0c7790 con 0x7f701c144b30 2024-01-16T15:00:19.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.320+0000 7f7021000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c0c79e0 con 0x7f701c0b1de0 2024-01-16T15:00:19.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.320+0000 7f7021000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f701c0c7cc0 con 0x7f701c15c410 2024-01-16T15:00:19.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.321+0000 7f7013e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f701c144b30 0x7f701c1f1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:19.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.321+0000 7f7013e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f701c144b30 0x7f701c1f1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45262/0 (socket says 172.21.15.143:45262) 2024-01-16T15:00:19.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.321+0000 7f7013e00640 1 -- 172.21.15.143:0/3223199000 learned_addr learned my addr 172.21.15.143:0/3223199000 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:19.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.321+0000 7f701b400640 1 --2- 172.21.15.143:0/3223199000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f701c0b1de0 0x7f701c15d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:19.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.321+0000 7f701be00640 1 --2- 172.21.15.143:0/3223199000 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f701c15c410 0x7f701c1403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:19.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.321+0000 7f7013e00640 1 -- 172.21.15.143:0/3223199000 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f701c15c410 msgr2=0x7f701c1403a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:19.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.321+0000 7f7013e00640 1 --2- 172.21.15.143:0/3223199000 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f701c15c410 0x7f701c1403a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:19.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.321+0000 7f7013e00640 1 -- 172.21.15.143:0/3223199000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f701c0b1de0 msgr2=0x7f701c15d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:19.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.321+0000 7f7013e00640 1 --2- 172.21.15.143:0/3223199000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f701c0b1de0 0x7f701c15d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:19.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.321+0000 7f7013e00640 1 -- 172.21.15.143:0/3223199000 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700c0094d0 con 0x7f701c144b30 2024-01-16T15:00:19.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.321+0000 7f7013e00640 1 --2- 172.21.15.143:0/3223199000 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f701c144b30 0x7f701c1f1d50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f700400c400 tx=0x7f700400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:19.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.321+0000 7f7013400640 1 -- 172.21.15.143:0/3223199000 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70040052d0 con 0x7f701c144b30 2024-01-16T15:00:19.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.322+0000 7f7013400640 1 -- 172.21.15.143:0/3223199000 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7004005430 con 0x7f701c144b30 2024-01-16T15:00:19.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.322+0000 7f7021000640 1 -- 172.21.15.143:0/3223199000 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f701c0c7fa0 con 0x7f701c144b30 2024-01-16T15:00:19.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.322+0000 7f7013400640 1 -- 172.21.15.143:0/3223199000 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7004003830 con 0x7f701c144b30 2024-01-16T15:00:19.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.322+0000 7f7021000640 1 -- 172.21.15.143:0/3223199000 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f701c15d640 con 0x7f701c144b30 2024-01-16T15:00:19.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.323+0000 7f7021000640 1 -- 172.21.15.143:0/3223199000 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f701c0c84c0 con 0x7f701c144b30 2024-01-16T15:00:19.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.324+0000 7f7013400640 1 -- 172.21.15.143:0/3223199000 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f7004003990 con 0x7f701c144b30 2024-01-16T15:00:19.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.325+0000 7f7013400640 1 --2- 172.21.15.143:0/3223199000 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6ff406b2c0 0x7f6ff406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:19.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.325+0000 7f7013400640 1 -- 172.21.15.143:0/3223199000 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7004089160 con 0x7f701c144b30 2024-01-16T15:00:19.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.325+0000 7f701b400640 1 --2- 172.21.15.143:0/3223199000 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6ff406b2c0 0x7f6ff406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:19.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.325+0000 7f701b400640 1 --2- 172.21.15.143:0/3223199000 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6ff406b2c0 0x7f6ff406d780 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f700c036000 tx=0x7f700c008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:19.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.329+0000 7f7013400640 1 -- 172.21.15.143:0/3223199000 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f701c0c84c0 con 0x7f701c144b30 2024-01-16T15:00:19.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.521+0000 7f7021000640 1 -- 172.21.15.143:0/3223199000 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f701c0c84c0 con 0x7f701c144b30 2024-01-16T15:00:19.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.524+0000 7f7013400640 1 -- 172.21.15.143:0/3223199000 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16565 (secure 0 0 0) 0x7f701c0c84c0 con 0x7f701c144b30 2024-01-16T15:00:19.506 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:00:19.506 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0","created":"2024-01-16T14:59:38.532601+0000","modified":"2024-01-16T15:00:04.543620+0000","last_up_change":"2024-01-16T14:59:50.983699+0000","last_in_change":"2024-01-16T14:59:42.056297+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:59:51.727720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T15:00:01.010472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T15:00:02.041228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f4c8891d-ab29-472a-907d-8446f20f7012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6801","nonce":1569293626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6805","nonce":1569293626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6814","nonce":1569293626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6809","nonce":1569293626}]},"public_addr":"172.21.15.143:6801/1569293626","cluster_addr":"172.21.15.143:6805/1569293626","heartbeat_back_addr":"172.21.15.143:6814/1569293626","heartbeat_front_addr":"172.21.15.143:6809/1569293626","state":["exists","up"]},{"osd":1,"uuid":"c4a2788d-8b89-4992-82b2-7f4b9debd509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6804","nonce":1276841168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6808","nonce":1276841168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6813","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6815","nonce":1276841168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6811","nonce":1276841168}]},"public_addr":"172.21.15.143:6804/1276841168","cluster_addr":"172.21.15.143:6808/1276841168","heartbeat_back_addr":"172.21.15.143:6815/1276841168","heartbeat_front_addr":"172.21.15.143:6811/1276841168","state":["exists","up"]},{"osd":2,"uuid":"64645c1d-31b2-4794-b0f2-7ccf8df30393","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6817","nonce":1217670170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6819","nonce":1217670170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6823","nonce":1217670170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6821","nonce":1217670170}]},"public_addr":"172.21.15.143:6817/1217670170","cluster_addr":"172.21.15.143:6819/1217670170","heartbeat_back_addr":"172.21.15.143:6823/1217670170","heartbeat_front_addr":"172.21.15.143:6821/1217670170","state":["exists","up"]},{"osd":3,"uuid":"73674129-a84e-4d31-8247-7d0179ae6df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6825","nonce":3414188012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6827","nonce":3414188012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6831","nonce":3414188012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6829","nonce":3414188012}]},"public_addr":"172.21.15.143:6825/3414188012","cluster_addr":"172.21.15.143:6827/3414188012","heartbeat_back_addr":"172.21.15.143:6831/3414188012","heartbeat_front_addr":"172.21.15.143:6829/3414188012","state":["exists","up"]},{"osd":4,"uuid":"cea28c9b-2f1c-4087-b415-9960789131d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6811","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6812","nonce":1070255716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6816","nonce":1070255716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6823","nonce":1070255716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6820","nonce":1070255716}]},"public_addr":"172.21.15.204:6812/1070255716","cluster_addr":"172.21.15.204:6816/1070255716","heartbeat_back_addr":"172.21.15.204:6823/1070255716","heartbeat_front_addr":"172.21.15.204:6820/1070255716","state":["exists","up"]},{"osd":5,"uuid":"33b63560-5574-4e78-983f-b398d67a2cb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6809","nonce":3605417837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6813","nonce":3605417837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6819","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6821","nonce":3605417837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6817","nonce":3605417837}]},"public_addr":"172.21.15.204:6809/3605417837","cluster_addr":"172.21.15.204:6813/3605417837","heartbeat_back_addr":"172.21.15.204:6821/3605417837","heartbeat_front_addr":"172.21.15.204:6817/3605417837","state":["exists","up"]},{"osd":6,"uuid":"df397427-8940-4eee-9c79-eece669d735d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6801","nonce":1085000534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6803","nonce":1085000534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6808","nonce":1085000534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6805","nonce":1085000534}]},"public_addr":"172.21.15.204:6801/1085000534","cluster_addr":"172.21.15.204:6803/1085000534","heartbeat_back_addr":"172.21.15.204:6808/1085000534","heartbeat_front_addr":"172.21.15.204:6805/1085000534","state":["exists","up"]},{"osd":7,"uuid":"a2e5a15c-8312-405a-ac02-f7e1fc9df2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6825","nonce":1337220609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6827","nonce":1337220609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6831","nonce":1337220609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6829","nonce":1337220609}]},"public_addr":"172.21.15.204:6825/1337220609","cluster_addr":"172.21.15.204:6827/1337220609","heartbeat_back_addr":"172.21.15.204:6831/1337220609","heartbeat_front_addr":"172.21.15.204:6829/1337220609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:49.984507+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.395306+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.632998+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.373627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.342920+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.438366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.331926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.406657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T15:00:19.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.527+0000 7f7021000640 1 -- 172.21.15.143:0/3223199000 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6ff406b2c0 msgr2=0x7f6ff406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:19.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.527+0000 7f7021000640 1 --2- 172.21.15.143:0/3223199000 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6ff406b2c0 0x7f6ff406d780 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f700c036000 tx=0x7f700c008030 comp rx=0 tx=0).stop 2024-01-16T15:00:19.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.527+0000 7f7021000640 1 -- 172.21.15.143:0/3223199000 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f701c144b30 msgr2=0x7f701c1f1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:19.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.527+0000 7f7021000640 1 --2- 172.21.15.143:0/3223199000 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f701c144b30 0x7f701c1f1d50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f700400c400 tx=0x7f700400c8c0 comp rx=0 tx=0).stop 2024-01-16T15:00:19.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.527+0000 7f7021000640 1 -- 172.21.15.143:0/3223199000 shutdown_connections 2024-01-16T15:00:19.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.527+0000 7f7021000640 1 --2- 172.21.15.143:0/3223199000 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6ff406b2c0 0x7f6ff406d780 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:19.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.527+0000 7f7021000640 1 --2- 172.21.15.143:0/3223199000 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f701c15c410 0x7f701c1403a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:19.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.527+0000 7f7021000640 1 --2- 172.21.15.143:0/3223199000 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f701c144b30 0x7f701c1f1d50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:19.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.527+0000 7f7021000640 1 --2- 172.21.15.143:0/3223199000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f701c0b1de0 0x7f701c15d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:19.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.527+0000 7f7021000640 1 -- 172.21.15.143:0/3223199000 >> 172.21.15.143:0/3223199000 conn(0x7f701c0cb830 msgr2=0x7f701c15e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:19.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.527+0000 7f7021000640 1 -- 172.21.15.143:0/3223199000 shutdown_connections 2024-01-16T15:00:19.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.527+0000 7f7021000640 1 -- 172.21.15.143:0/3223199000 wait complete. 2024-01-16T15:00:19.527 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-16T15:00:19.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.752+0000 7ff68f800640 1 Processor -- start 2024-01-16T15:00:19.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.752+0000 7ff68f800640 1 -- start start 2024-01-16T15:00:19.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.754+0000 7ff68f800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff68815df80 0x7ff6880ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:19.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.754+0000 7ff68f800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff68815e3f0 con 0x7ff6880b2250 2024-01-16T15:00:19.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.754+0000 7ff68f800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff68815ab80 con 0x7ff6880cd470 2024-01-16T15:00:19.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.754+0000 7ff68f800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff68815a990 con 0x7ff68815df80 2024-01-16T15:00:19.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.754+0000 7ff68ce00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff6880b2250 0x7ff6880b2630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:37990/0 (socket says 172.21.15.143:37990) 2024-01-16T15:00:19.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.754+0000 7ff68ce00640 1 -- 172.21.15.143:0/652579533 learned_addr learned my addr 172.21.15.143:0/652579533 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:19.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.754+0000 7ff687e00640 1 -- 172.21.15.143:0/652579533 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2031481667 0 0) 0x7ff68815e3f0 con 0x7ff6880b2250 2024-01-16T15:00:19.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.754+0000 7ff687e00640 1 -- 172.21.15.143:0/652579533 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff668003520 con 0x7ff6880b2250 2024-01-16T15:00:19.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.755+0000 7ff67fc00640 1 --2- 172.21.15.143:0/652579533 >> v2:172.21.15.204:3300/0 conn(0x7ff68815df80 0x7ff6880ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:19.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.755+0000 7ff687e00640 1 -- 172.21.15.143:0/652579533 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 546504802 0 0) 0x7ff668003520 con 0x7ff6880b2250 2024-01-16T15:00:19.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.755+0000 7ff687e00640 1 -- 172.21.15.143:0/652579533 >> v2:172.21.15.204:3300/0 conn(0x7ff68815df80 msgr2=0x7ff6880ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:19.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.755+0000 7ff687e00640 1 --2- 172.21.15.143:0/652579533 >> v2:172.21.15.204:3300/0 conn(0x7ff68815df80 0x7ff6880ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:19.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.755+0000 7ff687e00640 1 -- 172.21.15.143:0/652579533 >> v1:172.21.15.204:6789/0 conn(0x7ff6880cd470 legacy=0x7ff68815b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:19.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.755+0000 7ff687e00640 1 -- 172.21.15.143:0/652579533 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6880b2d40 con 0x7ff6880b2250 2024-01-16T15:00:19.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.755+0000 7ff687e00640 1 -- 172.21.15.143:0/652579533 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7ff678001670 con 0x7ff6880b2250 2024-01-16T15:00:19.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.755+0000 7ff687e00640 1 -- 172.21.15.143:0/652579533 >> v1:172.21.15.143:6789/0 conn(0x7ff6880b2250 legacy=0x7ff6880b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:19.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.755+0000 7ff687e00640 1 --2- 172.21.15.143:0/652579533 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6680039c0 0x7ff668003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:19.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.755+0000 7ff687e00640 1 --2- 172.21.15.143:0/652579533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6680047c0 0x7ff668004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:19.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.755+0000 7ff687e00640 1 --2- 172.21.15.143:0/652579533 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6680051d0 0x7ff66800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:19.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.755+0000 7ff687e00640 1 -- 172.21.15.143:0/652579533 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66800c0d0 con 0x7ff6680039c0 2024-01-16T15:00:19.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.755+0000 7ff687e00640 1 -- 172.21.15.143:0/652579533 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66800c240 con 0x7ff6680047c0 2024-01-16T15:00:19.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.755+0000 7ff687e00640 1 -- 172.21.15.143:0/652579533 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff66800c400 con 0x7ff6680051d0 2024-01-16T15:00:19.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.755+0000 7ff68ce00640 1 --2- 172.21.15.143:0/652579533 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6680039c0 0x7ff668003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:19.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.755+0000 7ff67fc00640 1 --2- 172.21.15.143:0/652579533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6680047c0 0x7ff668004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:19.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.755+0000 7ff68d800640 1 --2- 172.21.15.143:0/652579533 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6680051d0 0x7ff66800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:19.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.756+0000 7ff68ce00640 1 -- 172.21.15.143:0/652579533 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6680051d0 msgr2=0x7ff66800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:19.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.756+0000 7ff68ce00640 1 --2- 172.21.15.143:0/652579533 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6680051d0 0x7ff66800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:19.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.756+0000 7ff68ce00640 1 -- 172.21.15.143:0/652579533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6680047c0 msgr2=0x7ff668004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:19.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.756+0000 7ff68ce00640 1 --2- 172.21.15.143:0/652579533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6680047c0 0x7ff668004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:19.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.756+0000 7ff68ce00640 1 -- 172.21.15.143:0/652579533 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff668003520 con 0x7ff6680039c0 2024-01-16T15:00:19.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.756+0000 7ff68d800640 1 -- 172.21.15.143:0/652579533 reap_dead start 2024-01-16T15:00:19.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.757+0000 7ff68ce00640 1 --2- 172.21.15.143:0/652579533 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6680039c0 0x7ff668003dc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff678007fd0 tx=0x7ff6680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af94b4ad5973c3f2 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:19.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.757+0000 7ff687e00640 1 -- 172.21.15.143:0/652579533 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff678005ad0 con 0x7ff6680039c0 2024-01-16T15:00:19.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.757+0000 7ff687e00640 1 -- 172.21.15.143:0/652579533 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff678005c30 con 0x7ff6680039c0 2024-01-16T15:00:19.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.757+0000 7ff687e00640 1 -- 172.21.15.143:0/652579533 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67802c7c0 con 0x7ff6680039c0 2024-01-16T15:00:19.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.757+0000 7ff68f800640 1 -- 172.21.15.143:0/652579533 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6680039c0 msgr2=0x7ff668003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:19.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.757+0000 7ff68f800640 1 --2- 172.21.15.143:0/652579533 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6680039c0 0x7ff668003dc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff678007fd0 tx=0x7ff6680057f0 comp rx=0 tx=0).stop 2024-01-16T15:00:19.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.757+0000 7ff68f800640 1 -- 172.21.15.143:0/652579533 shutdown_connections 2024-01-16T15:00:19.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.757+0000 7ff68f800640 1 --2- 172.21.15.143:0/652579533 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6680039c0 0x7ff668003dc0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:19.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.757+0000 7ff68f800640 1 -- 172.21.15.143:0/652579533 >> 172.21.15.143:0/652579533 conn(0x7ff68806a920 msgr2=0x7ff68806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:19.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.758+0000 7ff68f800640 1 -- 172.21.15.143:0/652579533 shutdown_connections 2024-01-16T15:00:19.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.758+0000 7ff68f800640 1 -- 172.21.15.143:0/652579533 wait complete. 2024-01-16T15:00:19.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.759+0000 7ff68f800640 1 Processor -- start 2024-01-16T15:00:19.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.759+0000 7ff68f800640 1 -- start start 2024-01-16T15:00:19.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.760+0000 7ff68f800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6680039c0 0x7ff688151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:19.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.760+0000 7ff68f800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff68814ff10 0x7ff688148b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:19.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.760+0000 7ff68f800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff68815c8b0 0x7ff68814ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:19.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.760+0000 7ff68f800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff688149080 con 0x7ff68814ff10 2024-01-16T15:00:19.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.760+0000 7ff68f800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6881504d0 con 0x7ff68815c8b0 2024-01-16T15:00:19.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.760+0000 7ff68f800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6881507b0 con 0x7ff6680039c0 2024-01-16T15:00:19.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.761+0000 7ff67fc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff68814ff10 0x7ff688148b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:19.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.761+0000 7ff67fc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff68814ff10 0x7ff688148b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45282/0 (socket says 172.21.15.143:45282) 2024-01-16T15:00:19.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.761+0000 7ff67fc00640 1 -- 172.21.15.143:0/2053573097 learned_addr learned my addr 172.21.15.143:0/2053573097 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:19.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.761+0000 7ff68ce00640 1 --2- 172.21.15.143:0/2053573097 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6680039c0 0x7ff688151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:19.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.761+0000 7ff67fc00640 1 -- 172.21.15.143:0/2053573097 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6680039c0 msgr2=0x7ff688151570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:19.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.761+0000 7ff67fc00640 1 --2- 172.21.15.143:0/2053573097 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6680039c0 0x7ff688151570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:19.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.761+0000 7ff67fc00640 1 -- 172.21.15.143:0/2053573097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff68815c8b0 msgr2=0x7ff68814ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:19.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.761+0000 7ff67fc00640 1 --2- 172.21.15.143:0/2053573097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff68815c8b0 0x7ff68814ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:19.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.761+0000 7ff67fc00640 1 -- 172.21.15.143:0/2053573097 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff688150a90 con 0x7ff68814ff10 2024-01-16T15:00:19.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.761+0000 7ff67fc00640 1 --2- 172.21.15.143:0/2053573097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff68814ff10 0x7ff688148b40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff670009be0 tx=0x7ff670007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:19.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.761+0000 7ff685600640 1 -- 172.21.15.143:0/2053573097 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff670014610 con 0x7ff68814ff10 2024-01-16T15:00:19.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.761+0000 7ff685600640 1 -- 172.21.15.143:0/2053573097 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67000f050 con 0x7ff68814ff10 2024-01-16T15:00:19.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.761+0000 7ff685600640 1 -- 172.21.15.143:0/2053573097 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6700149b0 con 0x7ff68814ff10 2024-01-16T15:00:19.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.761+0000 7ff68f800640 1 -- 172.21.15.143:0/2053573097 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff688150d70 con 0x7ff68814ff10 2024-01-16T15:00:19.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.762+0000 7ff68f800640 1 -- 172.21.15.143:0/2053573097 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff68815cd60 con 0x7ff68814ff10 2024-01-16T15:00:19.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.763+0000 7ff68f800640 1 -- 172.21.15.143:0/2053573097 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff68815cef0 con 0x7ff68814ff10 2024-01-16T15:00:19.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.763+0000 7ff685600640 1 -- 172.21.15.143:0/2053573097 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7ff670014b10 con 0x7ff68814ff10 2024-01-16T15:00:19.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.764+0000 7ff685600640 1 --2- 172.21.15.143:0/2053573097 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7ff66006b2c0 0x7ff66006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:19.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.764+0000 7ff685600640 1 -- 172.21.15.143:0/2053573097 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff67008a400 con 0x7ff68814ff10 2024-01-16T15:00:19.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.764+0000 7ff68ce00640 1 --2- 172.21.15.143:0/2053573097 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7ff66006b2c0 0x7ff66006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:19.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.765+0000 7ff68ce00640 1 --2- 172.21.15.143:0/2053573097 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7ff66006b2c0 0x7ff66006d780 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff678034000 tx=0x7ff678009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:19.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.768+0000 7ff685600640 1 -- 172.21.15.143:0/2053573097 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff6700551d0 con 0x7ff68814ff10 2024-01-16T15:00:19.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.979+0000 7ff68f800640 1 -- 172.21.15.143:0/2053573097 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6880ce180 con 0x7ff68814ff10 2024-01-16T15:00:19.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.980+0000 7ff685600640 1 -- 172.21.15.143:0/2053573097 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4916 (secure 0 0 0) 0x7ff6880ce180 con 0x7ff68814ff10 2024-01-16T15:00:19.961 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:00:19.961 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3599169408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3599169408},{"type":"v1","addr":"172.21.15.204:6834","nonce":3599169408}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/859082411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":859082411},{"type":"v1","addr":"172.21.15.204:6835","nonce":859082411}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6836/1564738611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1564738611},{"type":"v1","addr":"172.21.15.143:6836","nonce":1564738611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-16T15:00:03.050082+0000","modified":"2024-01-16T15:00:06.109664+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/3120706283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6835","nonce":3120706283},{"type":"v1","addr":"172.21.15.143:6837","nonce":3120706283}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-16T15:00:19.961 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 8 2024-01-16T15:00:19.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.982+0000 7ff68f800640 1 -- 172.21.15.143:0/2053573097 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7ff66006b2c0 msgr2=0x7ff66006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:19.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.982+0000 7ff68f800640 1 --2- 172.21.15.143:0/2053573097 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7ff66006b2c0 0x7ff66006d780 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff678034000 tx=0x7ff678009030 comp rx=0 tx=0).stop 2024-01-16T15:00:19.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.982+0000 7ff68f800640 1 -- 172.21.15.143:0/2053573097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff68814ff10 msgr2=0x7ff688148b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:19.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.982+0000 7ff68f800640 1 --2- 172.21.15.143:0/2053573097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff68814ff10 0x7ff688148b40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff670009be0 tx=0x7ff670007700 comp rx=0 tx=0).stop 2024-01-16T15:00:19.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.982+0000 7ff68f800640 1 -- 172.21.15.143:0/2053573097 shutdown_connections 2024-01-16T15:00:19.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.982+0000 7ff68f800640 1 --2- 172.21.15.143:0/2053573097 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7ff66006b2c0 0x7ff66006d780 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:19.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.982+0000 7ff68f800640 1 --2- 172.21.15.143:0/2053573097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff68815c8b0 0x7ff68814ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:19.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.982+0000 7ff68f800640 1 --2- 172.21.15.143:0/2053573097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff68814ff10 0x7ff688148b40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:19.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.982+0000 7ff68f800640 1 --2- 172.21.15.143:0/2053573097 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6680039c0 0x7ff688151570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:19.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.982+0000 7ff68f800640 1 -- 172.21.15.143:0/2053573097 >> 172.21.15.143:0/2053573097 conn(0x7ff68806a920 msgr2=0x7ff6880cf420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:19.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.982+0000 7ff68f800640 1 -- 172.21.15.143:0/2053573097 shutdown_connections 2024-01-16T15:00:19.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:19.982+0000 7ff68f800640 1 -- 172.21.15.143:0/2053573097 wait complete. 2024-01-16T15:00:20.383 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-16T15:00:03.050082+0000', 'modified': '2024-01-16T15:00:06.109664+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4275}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4275': {'gid': 4275, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.143:6837/3120706283', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6835', 'nonce': 3120706283}, {'type': 'v1', 'addr': '172.21.15.143:6837', 'nonce': 3120706283}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-01-16T15:00:20.383 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-16T15:00:20.384 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-01-16T15:00:20.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.622+0000 7fc019e00640 1 Processor -- start 2024-01-16T15:00:20.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.622+0000 7fc019e00640 1 -- start start 2024-01-16T15:00:20.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.624+0000 7fc019e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc01415d490 0x7fc0140cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:20.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.624+0000 7fc019e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc0140d3940 0x7fc0140d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:20.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.624+0000 7fc019e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0140cea30 con 0x7fc0140b63d0 2024-01-16T15:00:20.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.624+0000 7fc019e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0140d0450 con 0x7fc0140d3940 2024-01-16T15:00:20.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.624+0000 7fc019e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0140d0bf0 con 0x7fc01415d490 2024-01-16T15:00:20.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.624+0000 7fc013e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc0140b63d0 0x7fc0140d3230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:37998/0 (socket says 172.21.15.143:37998) 2024-01-16T15:00:20.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.624+0000 7fc019400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc0140d3940 0x7fc0140d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:20.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.624+0000 7fc013e00640 1 -- 172.21.15.143:0/3141721468 learned_addr learned my addr 172.21.15.143:0/3141721468 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:20.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.624+0000 7fc018a00640 1 --2- 172.21.15.143:0/3141721468 >> v2:172.21.15.204:3300/0 conn(0x7fc01415d490 0x7fc0140cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:20.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.625+0000 7fc013400640 1 -- 172.21.15.143:0/3141721468 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2134541905 0 0) 0x7fc0140cea30 con 0x7fc0140b63d0 2024-01-16T15:00:20.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.625+0000 7fc013400640 1 -- 172.21.15.143:0/3141721468 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbff4003520 con 0x7fc0140b63d0 2024-01-16T15:00:20.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.625+0000 7fc019400640 1 -- 172.21.15.143:0/3141721468 >> v2:172.21.15.204:3300/0 conn(0x7fc01415d490 msgr2=0x7fc0140cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:20.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.625+0000 7fc019400640 1 --2- 172.21.15.143:0/3141721468 >> v2:172.21.15.204:3300/0 conn(0x7fc01415d490 0x7fc0140cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:20.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.625+0000 7fc019400640 1 -- 172.21.15.143:0/3141721468 >> v1:172.21.15.143:6789/0 conn(0x7fc0140b63d0 legacy=0x7fc0140d3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:20.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.625+0000 7fc019400640 1 -- 172.21.15.143:0/3141721468 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0140ce750 con 0x7fc0140d3940 2024-01-16T15:00:20.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.626+0000 7fc019400640 1 --2- 172.21.15.143:0/3141721468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0140d3940 0x7fc0140d1250 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc0140b52b0 tx=0x7fc00c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee4ec0412f4f555 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:20.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.626+0000 7fc013400640 1 -- 172.21.15.143:0/3141721468 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00c02cc80 con 0x7fc0140d3940 2024-01-16T15:00:20.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.626+0000 7fc013400640 1 -- 172.21.15.143:0/3141721468 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc00c02cde0 con 0x7fc0140d3940 2024-01-16T15:00:20.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.626+0000 7fc013400640 1 -- 172.21.15.143:0/3141721468 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00c034640 con 0x7fc0140d3940 2024-01-16T15:00:20.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.627+0000 7fc019e00640 1 -- 172.21.15.143:0/3141721468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0140d3940 msgr2=0x7fc0140d1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:20.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.627+0000 7fc019e00640 1 --2- 172.21.15.143:0/3141721468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0140d3940 0x7fc0140d1250 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc0140b52b0 tx=0x7fc00c02bed0 comp rx=0 tx=0).stop 2024-01-16T15:00:20.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.627+0000 7fc019e00640 1 -- 172.21.15.143:0/3141721468 shutdown_connections 2024-01-16T15:00:20.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.627+0000 7fc019e00640 1 --2- 172.21.15.143:0/3141721468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0140d3940 0x7fc0140d1250 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:20.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.627+0000 7fc019e00640 1 --2- 172.21.15.143:0/3141721468 >> v2:172.21.15.204:3300/0 conn(0x7fc01415d490 0x7fc0140cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:20.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.627+0000 7fc019e00640 1 -- 172.21.15.143:0/3141721468 >> 172.21.15.143:0/3141721468 conn(0x7fc014061340 msgr2=0x7fc014061750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:20.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.627+0000 7fc019e00640 1 -- 172.21.15.143:0/3141721468 shutdown_connections 2024-01-16T15:00:20.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.627+0000 7fc019e00640 1 -- 172.21.15.143:0/3141721468 wait complete. 2024-01-16T15:00:20.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.629+0000 7fc019e00640 1 Processor -- start 2024-01-16T15:00:20.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.629+0000 7fc019e00640 1 -- start start 2024-01-16T15:00:20.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.630+0000 7fc019e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0140d3940 0x7fc0141599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:20.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.630+0000 7fc019e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc01415d490 0x7fc0140cc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:20.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.630+0000 7fc019e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0140cc9a0 0x7fc0140c8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:20.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.630+0000 7fc019e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01415db70 con 0x7fc0140d3940 2024-01-16T15:00:20.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.630+0000 7fc019e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0140cb3c0 con 0x7fc0140cc9a0 2024-01-16T15:00:20.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.630+0000 7fc019e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0140cb6a0 con 0x7fc01415d490 2024-01-16T15:00:20.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.630+0000 7fc018a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0140d3940 0x7fc0141599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:20.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.630+0000 7fc018a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0140d3940 0x7fc0141599c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45302/0 (socket says 172.21.15.143:45302) 2024-01-16T15:00:20.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.630+0000 7fc018a00640 1 -- 172.21.15.143:0/1114843699 learned_addr learned my addr 172.21.15.143:0/1114843699 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:20.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.630+0000 7fc018a00640 1 -- 172.21.15.143:0/1114843699 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc01415d490 msgr2=0x7fc0140cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:20.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.630+0000 7fc018a00640 1 --2- 172.21.15.143:0/1114843699 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc01415d490 0x7fc0140cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:20.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.630+0000 7fc018a00640 1 -- 172.21.15.143:0/1114843699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0140cc9a0 msgr2=0x7fc0140c8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:20.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.630+0000 7fc019400640 1 --2- 172.21.15.143:0/1114843699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0140cc9a0 0x7fc0140c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:20.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.630+0000 7fc018a00640 1 --2- 172.21.15.143:0/1114843699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0140cc9a0 0x7fc0140c8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:20.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.630+0000 7fc018a00640 1 -- 172.21.15.143:0/1114843699 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc00c009a70 con 0x7fc0140d3940 2024-01-16T15:00:20.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.631+0000 7fc018a00640 1 --2- 172.21.15.143:0/1114843699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0140d3940 0x7fc0141599c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc00400e870 tx=0x7fc00400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:20.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.631+0000 7fc010c00640 1 -- 172.21.15.143:0/1114843699 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0040094f0 con 0x7fc0140d3940 2024-01-16T15:00:20.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.631+0000 7fc010c00640 1 -- 172.21.15.143:0/1114843699 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc00400c940 con 0x7fc0140d3940 2024-01-16T15:00:20.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.631+0000 7fc010c00640 1 -- 172.21.15.143:0/1114843699 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00400ccf0 con 0x7fc0140d3940 2024-01-16T15:00:20.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.631+0000 7fc019e00640 1 -- 172.21.15.143:0/1114843699 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0140cb980 con 0x7fc0140d3940 2024-01-16T15:00:20.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.631+0000 7fc019e00640 1 -- 172.21.15.143:0/1114843699 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0140cbf40 con 0x7fc0140d3940 2024-01-16T15:00:20.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.634+0000 7fc010c00640 1 -- 172.21.15.143:0/1114843699 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fc004013470 con 0x7fc0140d3940 2024-01-16T15:00:20.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.635+0000 7fc010c00640 1 --2- 172.21.15.143:0/1114843699 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbfec06b2c0 0x7fbfec06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:20.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.635+0000 7fc010c00640 1 -- 172.21.15.143:0/1114843699 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc004089cf0 con 0x7fc0140d3940 2024-01-16T15:00:20.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.635+0000 7fc013e00640 1 --2- 172.21.15.143:0/1114843699 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbfec06b2c0 0x7fbfec06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:20.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.635+0000 7fc01e176740 1 Processor -- start 2024-01-16T15:00:20.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.636+0000 7fc013e00640 1 --2- 172.21.15.143:0/1114843699 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbfec06b2c0 0x7fbfec06d780 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbffc000f60 tx=0x7fbffc0078f0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:20.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.636+0000 7fc01e176740 1 -- start start 2024-01-16T15:00:20.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.636+0000 7fc01e176740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a76248070 0x557a76248450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:20.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.636+0000 7fc01e176740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a76242d70 0x557a76247270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:20.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.636+0000 7fc01e176740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x557a762477b0 0x557a7628b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:20.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.636+0000 7fc01e176740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a761a6880 con 0x557a76242d70 2024-01-16T15:00:20.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.636+0000 7fc01e176740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a761b5ae0 con 0x557a76248070 2024-01-16T15:00:20.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.637+0000 7fc01e176740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x557a761b31d0 con 0x557a762477b0 2024-01-16T15:00:20.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.637+0000 7fc013e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a76242d70 0x557a76247270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:20.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.637+0000 7fc013e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a76242d70 0x557a76247270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45316/0 (socket says 172.21.15.143:45316) 2024-01-16T15:00:20.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.637+0000 7fc013e00640 1 -- 172.21.15.143:0/3986343994 learned_addr learned my addr 172.21.15.143:0/3986343994 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:20.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.637+0000 7fc019400640 1 --2- 172.21.15.143:0/3986343994 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x557a762477b0 0x557a7628b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:20.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.637+0000 7fc013e00640 1 -- 172.21.15.143:0/3986343994 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x557a762477b0 msgr2=0x557a7628b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:20.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.637+0000 7fc018a00640 1 --2- 172.21.15.143:0/3986343994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a76248070 0x557a76248450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:20.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.637+0000 7fc013e00640 1 --2- 172.21.15.143:0/3986343994 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x557a762477b0 0x557a7628b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:20.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.637+0000 7fc013e00640 1 -- 172.21.15.143:0/3986343994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a76248070 msgr2=0x557a76248450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:20.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.637+0000 7fc013e00640 1 --2- 172.21.15.143:0/3986343994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a76248070 0x557a76248450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:20.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.637+0000 7fc013e00640 1 -- 172.21.15.143:0/3986343994 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a76176860 con 0x557a76242d70 2024-01-16T15:00:20.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.637+0000 7fc013e00640 1 --2- 172.21.15.143:0/3986343994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a76242d70 0x557a76247270 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbffc005c30 tx=0x7fbffc00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca951860daada70 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:20.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.638+0000 7fc009600640 1 -- 172.21.15.143:0/3986343994 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc01b070 con 0x557a76242d70 2024-01-16T15:00:20.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.638+0000 7fc009600640 1 -- 172.21.15.143:0/3986343994 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbffc016110 con 0x557a76242d70 2024-01-16T15:00:20.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.638+0000 7fc009600640 1 -- 172.21.15.143:0/3986343994 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc005e60 con 0x557a76242d70 2024-01-16T15:00:20.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.638+0000 7fc01e176740 1 -- 172.21.15.143:0/3986343994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a76242d70 msgr2=0x557a76247270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:20.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.638+0000 7fc01e176740 1 --2- 172.21.15.143:0/3986343994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a76242d70 0x557a76247270 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbffc005c30 tx=0x7fbffc00f000 comp rx=0 tx=0).stop 2024-01-16T15:00:20.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.638+0000 7fc01e176740 1 -- 172.21.15.143:0/3986343994 shutdown_connections 2024-01-16T15:00:20.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.638+0000 7fc01e176740 1 --2- 172.21.15.143:0/3986343994 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x557a762477b0 0x557a7628b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:20.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.638+0000 7fc01e176740 1 --2- 172.21.15.143:0/3986343994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a76242d70 0x557a76247270 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:20.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.638+0000 7fc01e176740 1 --2- 172.21.15.143:0/3986343994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a76248070 0x557a76248450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:20.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.638+0000 7fc01e176740 1 -- 172.21.15.143:0/3986343994 >> 172.21.15.143:0/3986343994 conn(0x557a76209c70 msgr2=0x557a7628db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:20.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.639+0000 7fc01e176740 1 -- 172.21.15.143:0/3986343994 shutdown_connections 2024-01-16T15:00:20.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.639+0000 7fc01e176740 1 -- 172.21.15.143:0/3986343994 wait complete. 2024-01-16T15:00:20.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.639+0000 7fc01e176740 1 -- start start 2024-01-16T15:00:20.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.639+0000 7fc01e176740 1 Processor -- start 2024-01-16T15:00:20.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.640+0000 7fc01e176740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a76242d70 0x557a76295000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:20.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.640+0000 7fc01e176740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a762477b0 0x557a76284fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:20.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.640+0000 7fc01e176740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x557a76248070 0x557a76285510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:20.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.640+0000 7fc01e176740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a761aeac0 con 0x557a76242d70 2024-01-16T15:00:20.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.640+0000 7fc01e176740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a7614b2f0 con 0x557a762477b0 2024-01-16T15:00:20.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.640+0000 7fc01e176740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x557a76168c70 con 0x557a76248070 2024-01-16T15:00:20.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.640+0000 7fc01e176740 10 client.0 _ll_register_callbacks cb 0x557a7616d3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-16T15:00:20.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.641+0000 7fc018a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a76242d70 0x557a76295000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:20.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.641+0000 7fc018a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a76242d70 0x557a76295000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45320/0 (socket says 172.21.15.143:45320) 2024-01-16T15:00:20.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.641+0000 7fc018a00640 1 -- 172.21.15.143:0/1406488110 learned_addr learned my addr 172.21.15.143:0/1406488110 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:20.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.641+0000 7fc018a00640 1 -- 172.21.15.143:0/1406488110 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x557a76248070 msgr2=0x557a76285510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:20.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.641+0000 7fc019400640 1 --2- 172.21.15.143:0/1406488110 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x557a76248070 0x557a76285510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:20.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.641+0000 7fc018a00640 1 --2- 172.21.15.143:0/1406488110 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x557a76248070 0x557a76285510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:20.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.641+0000 7fc018a00640 1 -- 172.21.15.143:0/1406488110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a762477b0 msgr2=0x557a76284fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:20.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.641+0000 7fc018a00640 1 --2- 172.21.15.143:0/1406488110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a762477b0 0x557a76284fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:20.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.641+0000 7fc018a00640 1 -- 172.21.15.143:0/1406488110 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff4003520 con 0x557a76242d70 2024-01-16T15:00:20.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.641+0000 7fc018a00640 1 --2- 172.21.15.143:0/1406488110 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a76242d70 0x557a76295000 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc0040605f0 tx=0x7fc00405c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13dd8a259bbb755e server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:20.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.641+0000 7fc002000640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-01-16T15:00:20.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.641+0000 7fc002000640 1 -- 172.21.15.143:0/1406488110 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc004060890 con 0x557a76242d70 2024-01-16T15:00:20.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.641+0000 7fc002000640 1 -- 172.21.15.143:0/1406488110 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc004059b50 con 0x557a76242d70 2024-01-16T15:00:20.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.641+0000 7fc01e176740 1 -- 172.21.15.143:0/1406488110 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a761a7aa0 con 0x557a76242d70 2024-01-16T15:00:20.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.642+0000 7fc002000640 1 -- 172.21.15.143:0/1406488110 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc004059d10 con 0x557a76242d70 2024-01-16T15:00:20.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.642+0000 7fc01e176740 10 client.4668 fetch_fsmap learned FSMap version 8 2024-01-16T15:00:20.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.642+0000 7fc01e176740 1 -- 172.21.15.143:0/1406488110 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x557a761ad140 con 0x557a76242d70 2024-01-16T15:00:20.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.642+0000 7fc002000640 1 -- 172.21.15.143:0/1406488110 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557a761a7aa0 con 0x557a76242d70 2024-01-16T15:00:20.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.642+0000 7fc002000640 1 -- 172.21.15.143:0/1406488110 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc004011070 con 0x557a76242d70 2024-01-16T15:00:20.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.642+0000 7fc01e176740 10 client.4668 fetch_fsmap finished waiting for FSMap version 8 2024-01-16T15:00:20.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.642+0000 7fc01e176740 10 client.4668 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-16T15:00:20.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.642+0000 7fc01e176740 20 client.4668 populate_metadata read hostname 'smithi143' 2024-01-16T15:00:20.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.642+0000 7fc01e176740 1 --2- 172.21.15.143:0/1406488110 >> [v2:172.21.15.143:6835/3120706283,v1:172.21.15.143:6837/3120706283] conn(0x557a7626d790 0x557a762aaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:20.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.642+0000 7fc01e176740 4 client.4668 mds_command: new command op to 4275 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-16T15:00:20.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.642+0000 7fc01e176740 1 -- 172.21.15.143:0/1406488110 --> [v2:172.21.15.143:6835/3120706283,v1:172.21.15.143:6837/3120706283] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a761a77c0 con 0x557a7626d790 2024-01-16T15:00:20.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.642+0000 7fc013e00640 1 --2- 172.21.15.143:0/1406488110 >> [v2:172.21.15.143:6835/3120706283,v1:172.21.15.143:6837/3120706283] conn(0x557a7626d790 0x557a762aaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:20.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.643+0000 7fc013e00640 1 --2- 172.21.15.143:0/1406488110 >> [v2:172.21.15.143:6835/3120706283,v1:172.21.15.143:6837/3120706283] conn(0x557a7626d790 0x557a762aaab0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8fc3ca46bf46f94 server_cookie=62e03de3d47efbed in_seq=0 out_seq=0 2024-01-16T15:00:20.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.643+0000 7fc002000640 10 client.4668 ms_handle_connect on v2:172.21.15.143:6835/3120706283 2024-01-16T15:00:20.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.643+0000 7fc002000640 1 -- 172.21.15.143:0/1406488110 <== mds.0 v2:172.21.15.143:6835/3120706283 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557a761a77c0 con 0x557a7626d790 2024-01-16T15:00:20.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.643+0000 7fc002000640 10 client.4668 handle_command_reply: tid=0 2024-01-16T15:00:20.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.643+0000 7fc01e176740 1 client.4668 shutdown 2024-01-16T15:00:20.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.644+0000 7fc01e176740 1 -- 172.21.15.143:0/1406488110 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a76242d70 msgr2=0x557a76295000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:20.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.644+0000 7fc01e176740 1 --2- 172.21.15.143:0/1406488110 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a76242d70 0x557a76295000 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc0040605f0 tx=0x7fc00405c970 comp rx=0 tx=0).stop 2024-01-16T15:00:20.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.644+0000 7fc01e176740 1 -- 172.21.15.143:0/1406488110 shutdown_connections 2024-01-16T15:00:20.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.644+0000 7fc01e176740 1 --2- 172.21.15.143:0/1406488110 >> [v2:172.21.15.143:6835/3120706283,v1:172.21.15.143:6837/3120706283] conn(0x557a7626d790 0x557a762aaab0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:20.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.644+0000 7fc01e176740 1 --2- 172.21.15.143:0/1406488110 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x557a76248070 0x557a76285510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:20.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.644+0000 7fc01e176740 1 --2- 172.21.15.143:0/1406488110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a762477b0 0x557a76284fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:20.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.644+0000 7fc01e176740 1 --2- 172.21.15.143:0/1406488110 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a76242d70 0x557a76295000 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:20.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.644+0000 7fc002000640 0 client.4668 ms_handle_reset on v2:172.21.15.143:6835/3120706283 2024-01-16T15:00:20.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.644+0000 7fc01e176740 1 -- 172.21.15.143:0/1406488110 >> 172.21.15.143:0/1406488110 conn(0x557a76209c70 msgr2=0x557a76248450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:20.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.644+0000 7fc01e176740 1 -- 172.21.15.143:0/1406488110 shutdown_connections 2024-01-16T15:00:20.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.644+0000 7fc01e176740 1 -- 172.21.15.143:0/1406488110 wait complete. 2024-01-16T15:00:20.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.644+0000 7fc01e176740 20 client.4668 trim_cache size 0 max 16384 2024-01-16T15:00:20.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.661+0000 7fc01e176740 1 Processor -- start 2024-01-16T15:00:20.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.661+0000 7fc01e176740 1 -- start start 2024-01-16T15:00:20.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.662+0000 7fc01e176740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a76242d70 0x557a76246c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:20.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.662+0000 7fc01e176740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a762477b0 0x557a76292420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:20.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.662+0000 7fc01e176740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x557a76248070 0x557a7626a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:20.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.662+0000 7fc01e176740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a75f11f20 con 0x557a762477b0 2024-01-16T15:00:20.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.662+0000 7fc01e176740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a76296260 con 0x557a76242d70 2024-01-16T15:00:20.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.662+0000 7fc01e176740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x557a76274dc0 con 0x557a76248070 2024-01-16T15:00:20.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.662+0000 7fc013e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a762477b0 0x557a76292420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:20.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.662+0000 7fc013e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a762477b0 0x557a76292420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45330/0 (socket says 172.21.15.143:45330) 2024-01-16T15:00:20.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.662+0000 7fc013e00640 1 -- 172.21.15.143:0/2021982188 learned_addr learned my addr 172.21.15.143:0/2021982188 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:20.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.662+0000 7fc013e00640 1 -- 172.21.15.143:0/2021982188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x557a76248070 msgr2=0x557a7626a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:20.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.662+0000 7fc013e00640 1 --2- 172.21.15.143:0/2021982188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x557a76248070 0x557a7626a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:20.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.662+0000 7fc013e00640 1 -- 172.21.15.143:0/2021982188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a76242d70 msgr2=0x557a76246c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:20.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.662+0000 7fc013e00640 1 --2- 172.21.15.143:0/2021982188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a76242d70 0x557a76246c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:20.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.662+0000 7fc013e00640 1 -- 172.21.15.143:0/2021982188 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a7614ab80 con 0x557a762477b0 2024-01-16T15:00:20.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.662+0000 7fc013e00640 1 --2- 172.21.15.143:0/2021982188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a762477b0 0x557a76292420 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbffc016820 tx=0x7fbffc01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4faa1a8f12c432a6 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:20.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.663+0000 7fc00a000640 1 -- 172.21.15.143:0/2021982188 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc005e30 con 0x557a762477b0 2024-01-16T15:00:20.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.663+0000 7fc00a000640 1 -- 172.21.15.143:0/2021982188 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbffc028b70 con 0x557a762477b0 2024-01-16T15:00:20.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.663+0000 7fc00a000640 1 -- 172.21.15.143:0/2021982188 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc016950 con 0x557a762477b0 2024-01-16T15:00:20.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.663+0000 7fc01e176740 1 -- 172.21.15.143:0/2021982188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a762477b0 msgr2=0x557a76292420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:20.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.663+0000 7fc01e176740 1 --2- 172.21.15.143:0/2021982188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a762477b0 0x557a76292420 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbffc016820 tx=0x7fbffc01b000 comp rx=0 tx=0).stop 2024-01-16T15:00:20.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.663+0000 7fc01e176740 1 -- 172.21.15.143:0/2021982188 shutdown_connections 2024-01-16T15:00:20.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.663+0000 7fc01e176740 1 --2- 172.21.15.143:0/2021982188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x557a76248070 0x557a7626a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:20.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.663+0000 7fc01e176740 1 --2- 172.21.15.143:0/2021982188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a762477b0 0x557a76292420 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:20.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.663+0000 7fc01e176740 1 --2- 172.21.15.143:0/2021982188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a76242d70 0x557a76246c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:20.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.663+0000 7fc01e176740 1 -- 172.21.15.143:0/2021982188 >> 172.21.15.143:0/2021982188 conn(0x557a76209c70 msgr2=0x557a76241dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:20.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.663+0000 7fc01e176740 1 -- 172.21.15.143:0/2021982188 shutdown_connections 2024-01-16T15:00:20.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.663+0000 7fc01e176740 1 -- 172.21.15.143:0/2021982188 wait complete. 2024-01-16T15:00:20.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.664+0000 7fc01e176740 1 -- start start 2024-01-16T15:00:20.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.664+0000 7fc01e176740 1 Processor -- start 2024-01-16T15:00:20.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.665+0000 7fc01e176740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a76242d70 0x557a7628a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:20.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.665+0000 7fc01e176740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x557a762477b0 0x557a762848c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:20.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.665+0000 7fc01e176740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a76248070 0x557a76272480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:20.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.665+0000 7fc01e176740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a7628a590 con 0x557a76248070 2024-01-16T15:00:20.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.665+0000 7fc01e176740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a76285e00 con 0x557a76242d70 2024-01-16T15:00:20.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.665+0000 7fc01e176740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x557a76273280 con 0x557a762477b0 2024-01-16T15:00:20.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.665+0000 7fc01e176740 10 client.0 _ll_register_callbacks cb 0x7fc004058f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-16T15:00:20.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.665+0000 7fc019400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a76248070 0x557a76272480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:20.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.666+0000 7fc019400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a76248070 0x557a76272480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45336/0 (socket says 172.21.15.143:45336) 2024-01-16T15:00:20.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.666+0000 7fc019400640 1 -- 172.21.15.143:0/172453730 learned_addr learned my addr 172.21.15.143:0/172453730 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:20.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.666+0000 7fc013e00640 1 --2- 172.21.15.143:0/172453730 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x557a762477b0 0x557a762848c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:20.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.666+0000 7fc018a00640 1 --2- 172.21.15.143:0/172453730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a76242d70 0x557a7628a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:20.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.666+0000 7fc019400640 1 -- 172.21.15.143:0/172453730 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x557a762477b0 msgr2=0x557a762848c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:20.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.666+0000 7fc019400640 1 --2- 172.21.15.143:0/172453730 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x557a762477b0 0x557a762848c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:20.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.666+0000 7fc019400640 1 -- 172.21.15.143:0/172453730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a76242d70 msgr2=0x557a7628a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:20.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.666+0000 7fc019400640 1 --2- 172.21.15.143:0/172453730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a76242d70 0x557a7628a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:20.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.666+0000 7fc019400640 1 -- 172.21.15.143:0/172453730 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a76176860 con 0x557a76248070 2024-01-16T15:00:20.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.666+0000 7fc019400640 1 --2- 172.21.15.143:0/172453730 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a76248070 0x557a76272480 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc00c02c4e0 tx=0x7fc00c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b27498763fbb79ff server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:20.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.666+0000 7fc002000640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-01-16T15:00:20.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.666+0000 7fc002000640 1 -- 172.21.15.143:0/172453730 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00c0348f0 con 0x557a76248070 2024-01-16T15:00:20.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.667+0000 7fc002000640 1 -- 172.21.15.143:0/172453730 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc00c049ae0 con 0x557a76248070 2024-01-16T15:00:20.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.667+0000 7fc01e176740 1 -- 172.21.15.143:0/172453730 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbffc016110 con 0x557a76248070 2024-01-16T15:00:20.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.667+0000 7fc002000640 1 -- 172.21.15.143:0/172453730 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc00c049ca0 con 0x557a76248070 2024-01-16T15:00:20.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.667+0000 7fc01e176740 10 client.4674 fetch_fsmap learned FSMap version 8 2024-01-16T15:00:20.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.667+0000 7fc01e176740 1 -- 172.21.15.143:0/172453730 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x557a76170170 con 0x557a76248070 2024-01-16T15:00:20.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.667+0000 7fc002000640 1 -- 172.21.15.143:0/172453730 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbffc016110 con 0x557a76248070 2024-01-16T15:00:20.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.667+0000 7fc002000640 1 -- 172.21.15.143:0/172453730 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc00c034a50 con 0x557a76248070 2024-01-16T15:00:20.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.667+0000 7fc01e176740 10 client.4674 fetch_fsmap finished waiting for FSMap version 8 2024-01-16T15:00:20.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.667+0000 7fc01e176740 10 client.4674 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-16T15:00:20.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.667+0000 7fc01e176740 20 client.4674 populate_metadata read hostname 'smithi143' 2024-01-16T15:00:20.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.667+0000 7fc01e176740 1 --2- 172.21.15.143:0/172453730 >> [v2:172.21.15.143:6835/3120706283,v1:172.21.15.143:6837/3120706283] conn(0x557a76246b00 0x557a76246ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:20.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.667+0000 7fc01e176740 4 client.4674 mds_command: new command op to 4275 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-16T15:00:20.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.667+0000 7fc01e176740 1 -- 172.21.15.143:0/172453730 --> [v2:172.21.15.143:6835/3120706283,v1:172.21.15.143:6837/3120706283] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557a75eaf6c0 con 0x557a76246b00 2024-01-16T15:00:20.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.667+0000 7fc018a00640 1 --2- 172.21.15.143:0/172453730 >> [v2:172.21.15.143:6835/3120706283,v1:172.21.15.143:6837/3120706283] conn(0x557a76246b00 0x557a76246ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:20.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.668+0000 7fc018a00640 1 --2- 172.21.15.143:0/172453730 >> [v2:172.21.15.143:6835/3120706283,v1:172.21.15.143:6837/3120706283] conn(0x557a76246b00 0x557a76246ee0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fc84b5fee78ae79 server_cookie=43cb107e02248a8f in_seq=0 out_seq=0 2024-01-16T15:00:20.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.668+0000 7fc002000640 10 client.4674 ms_handle_connect on v2:172.21.15.143:6835/3120706283 2024-01-16T15:00:20.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.668+0000 7fc002000640 1 -- 172.21.15.143:0/172453730 <== mds.0 v2:172.21.15.143:6835/3120706283 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x557a75eaf6c0 con 0x557a76246b00 2024-01-16T15:00:20.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.668+0000 7fc002000640 10 client.4674 handle_command_reply: tid=0 2024-01-16T15:00:20.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.668+0000 7fc01e176740 1 client.4674 shutdown 2024-01-16T15:00:20.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.668+0000 7fc01e176740 1 -- 172.21.15.143:0/172453730 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a76248070 msgr2=0x557a76272480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:20.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.668+0000 7fc01e176740 1 --2- 172.21.15.143:0/172453730 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a76248070 0x557a76272480 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc00c02c4e0 tx=0x7fc00c004a40 comp rx=0 tx=0).stop 2024-01-16T15:00:20.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.669+0000 7fc01e176740 1 -- 172.21.15.143:0/172453730 shutdown_connections 2024-01-16T15:00:20.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.669+0000 7fc01e176740 1 --2- 172.21.15.143:0/172453730 >> [v2:172.21.15.143:6835/3120706283,v1:172.21.15.143:6837/3120706283] conn(0x557a76246b00 0x557a76246ee0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:20.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.669+0000 7fc01e176740 1 --2- 172.21.15.143:0/172453730 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a76248070 0x557a76272480 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:20.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.669+0000 7fc01e176740 1 --2- 172.21.15.143:0/172453730 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x557a762477b0 0x557a762848c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:20.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.669+0000 7fc01e176740 1 --2- 172.21.15.143:0/172453730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a76242d70 0x557a7628a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:20.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.669+0000 7fc002000640 0 client.4674 ms_handle_reset on v2:172.21.15.143:6835/3120706283 2024-01-16T15:00:20.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.669+0000 7fc01e176740 1 -- 172.21.15.143:0/172453730 >> 172.21.15.143:0/172453730 conn(0x557a76209c70 msgr2=0x557a762af120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:20.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.669+0000 7fc01e176740 1 -- 172.21.15.143:0/172453730 shutdown_connections 2024-01-16T15:00:20.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.669+0000 7fc01e176740 1 -- 172.21.15.143:0/172453730 wait complete. 2024-01-16T15:00:20.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.669+0000 7fc01e176740 20 client.4674 trim_cache size 0 max 16384 2024-01-16T15:00:20.655 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-16T15:00:20.655 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0", 2024-01-16T15:00:20.655 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 0, 2024-01-16T15:00:20.655 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 4275, 2024-01-16T15:00:20.655 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2024-01-16T15:00:20.655 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2024-01-16T15:00:20.655 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2024-01-16T15:00:20.655 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 17.073298303000001, 2024-01-16T15:00:20.655 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 8, 2024-01-16T15:00:20.655 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 20, 2024-01-16T15:00:20.655 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 20, 2024-01-16T15:00:20.655 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 22.567504161999999 2024-01-16T15:00:20.656 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-16T15:00:20.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.670+0000 7fc019e00640 1 -- 172.21.15.143:0/1114843699 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbfec06b2c0 msgr2=0x7fbfec06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:20.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.670+0000 7fc019e00640 1 --2- 172.21.15.143:0/1114843699 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbfec06b2c0 0x7fbfec06d780 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbffc000f60 tx=0x7fbffc0078f0 comp rx=0 tx=0).stop 2024-01-16T15:00:20.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.670+0000 7fc019e00640 1 -- 172.21.15.143:0/1114843699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0140d3940 msgr2=0x7fc0141599c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:20.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.670+0000 7fc019e00640 1 --2- 172.21.15.143:0/1114843699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0140d3940 0x7fc0141599c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc00400e870 tx=0x7fc00400ed30 comp rx=0 tx=0).stop 2024-01-16T15:00:20.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.670+0000 7fc019e00640 1 -- 172.21.15.143:0/1114843699 shutdown_connections 2024-01-16T15:00:20.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.670+0000 7fc019e00640 1 --2- 172.21.15.143:0/1114843699 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbfec06b2c0 0x7fbfec06d780 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:20.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.670+0000 7fc019e00640 1 --2- 172.21.15.143:0/1114843699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0140cc9a0 0x7fc0140c8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:20.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.670+0000 7fc019e00640 1 --2- 172.21.15.143:0/1114843699 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc01415d490 0x7fc0140cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:20.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.670+0000 7fc019e00640 1 --2- 172.21.15.143:0/1114843699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0140d3940 0x7fc0141599c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:20.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.670+0000 7fc019e00640 1 -- 172.21.15.143:0/1114843699 >> 172.21.15.143:0/1114843699 conn(0x7fc014061340 msgr2=0x7fc0140b3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:20.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.670+0000 7fc019e00640 1 -- 172.21.15.143:0/1114843699 shutdown_connections 2024-01-16T15:00:20.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:20.670+0000 7fc019e00640 1 -- 172.21.15.143:0/1114843699 wait complete. 2024-01-16T15:00:20.667 INFO:teuthology.run_tasks:Running task kclient... 2024-01-16T15:00:20.689 INFO:tasks.kclient:Mounting kernel clients... 2024-01-16T15:00:20.689 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-01-16T15:00:20.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:20.689 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2024-01-16T15:00:20.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:20.734 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link delete ceph-brx 2024-01-16T15:00:20.873 INFO:teuthology.orchestra.run.smithi143.stderr:Cannot find device "ceph-brx" 2024-01-16T15:00:20.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-16T15:00:20.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:20.876 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2024-01-16T15:00:20.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:20.927 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link delete ceph-brx 2024-01-16T15:00:21.014 INFO:teuthology.orchestra.run.smithi143.stderr:Cannot find device "ceph-brx" 2024-01-16T15:00:21.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-16T15:00:21.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:21.017 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-01-16T15:00:21.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:21.078 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link delete ceph-brx 2024-01-16T15:00:21.177 INFO:teuthology.orchestra.run.smithi204.stderr:Cannot find device "ceph-brx" 2024-01-16T15:00:21.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-16T15:00:21.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:21.181 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-01-16T15:00:21.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:21.233 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link delete ceph-brx 2024-01-16T15:00:21.328 INFO:teuthology.orchestra.run.smithi204.stderr:Cannot find device "ceph-brx" 2024-01-16T15:00:21.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-16T15:00:21.332 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-01-16T15:00:21.332 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-01-16T15:00:21.332 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-16T15:00:21.332 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-16T15:00:21.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.565+0000 7f3151c00640 1 Processor -- start 2024-01-16T15:00:21.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.565+0000 7f3151c00640 1 -- start start 2024-01-16T15:00:21.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.567+0000 7f3151c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f314c1461d0 0x7f314c0b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:21.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.567+0000 7f3151c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f314c0b4fe0 0x7f314c0b3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:21.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.567+0000 7f3151c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f314c06d180 con 0x7f314c0b3890 2024-01-16T15:00:21.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.567+0000 7f3151c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f314c0b3d00 con 0x7f314c1461d0 2024-01-16T15:00:21.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.567+0000 7f3151c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f314c06ba70 con 0x7f314c0b4fe0 2024-01-16T15:00:21.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.568+0000 7f3143e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f314c0b4fe0 0x7f314c0b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:21.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.568+0000 7f314aa00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f314c1461d0 0x7f314c0b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:21.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.568+0000 7f3143e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f314c0b4fe0 0x7f314c0b3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:35276/0 (socket says 172.21.15.143:35276) 2024-01-16T15:00:21.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.568+0000 7f3143e00640 1 -- 172.21.15.143:0/2387514719 learned_addr learned my addr 172.21.15.143:0/2387514719 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:21.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.568+0000 7f3143e00640 1 -- 172.21.15.143:0/2387514719 >> v2:172.21.15.204:3300/0 conn(0x7f314c1461d0 msgr2=0x7f314c0b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:21.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.568+0000 7f3143e00640 1 --2- 172.21.15.143:0/2387514719 >> v2:172.21.15.204:3300/0 conn(0x7f314c1461d0 0x7f314c0b4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:21.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.568+0000 7f3143e00640 1 -- 172.21.15.143:0/2387514719 >> v1:172.21.15.204:6789/0 conn(0x7f314c0b3890 legacy=0x7f314c15dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:21.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.568+0000 7f3143e00640 1 -- 172.21.15.143:0/2387514719 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f314c06bd50 con 0x7f314c0b4fe0 2024-01-16T15:00:21.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.568+0000 7f314aa00640 1 --2- 172.21.15.143:0/2387514719 >> v2:172.21.15.204:3300/0 conn(0x7f314c1461d0 0x7f314c0b4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T15:00:21.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.570+0000 7f3143e00640 1 --2- 172.21.15.143:0/2387514719 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f314c0b4fe0 0x7f314c0b3350 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3134009870 tx=0x7f313402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4346947810619e4b server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:21.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.570+0000 7f314a000640 1 -- 172.21.15.143:0/2387514719 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3134030840 con 0x7f314c0b4fe0 2024-01-16T15:00:21.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.570+0000 7f314a000640 1 -- 172.21.15.143:0/2387514719 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31340309a0 con 0x7f314c0b4fe0 2024-01-16T15:00:21.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.570+0000 7f314a000640 1 -- 172.21.15.143:0/2387514719 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3134030d30 con 0x7f314c0b4fe0 2024-01-16T15:00:21.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.571+0000 7f3151c00640 1 -- 172.21.15.143:0/2387514719 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f314c0b4fe0 msgr2=0x7f314c0b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:21.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.571+0000 7f3151c00640 1 --2- 172.21.15.143:0/2387514719 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f314c0b4fe0 0x7f314c0b3350 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3134009870 tx=0x7f313402b980 comp rx=0 tx=0).stop 2024-01-16T15:00:21.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.571+0000 7f3151c00640 1 -- 172.21.15.143:0/2387514719 shutdown_connections 2024-01-16T15:00:21.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.571+0000 7f3151c00640 1 --2- 172.21.15.143:0/2387514719 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f314c0b4fe0 0x7f314c0b3350 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:21.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.571+0000 7f3151c00640 1 --2- 172.21.15.143:0/2387514719 >> v2:172.21.15.204:3300/0 conn(0x7f314c1461d0 0x7f314c0b4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:21.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.571+0000 7f3151c00640 1 -- 172.21.15.143:0/2387514719 >> 172.21.15.143:0/2387514719 conn(0x7f314c061340 msgr2=0x7f314c061750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:21.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.571+0000 7f3151c00640 1 -- 172.21.15.143:0/2387514719 shutdown_connections 2024-01-16T15:00:21.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.571+0000 7f3151c00640 1 -- 172.21.15.143:0/2387514719 wait complete. 2024-01-16T15:00:21.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.573+0000 7f3151c00640 1 Processor -- start 2024-01-16T15:00:21.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.573+0000 7f3151c00640 1 -- start start 2024-01-16T15:00:21.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.574+0000 7f3151c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f314c0b3890 0x7f314c15c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:21.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.574+0000 7f3151c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f314c0b4fe0 0x7f314c0ce600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:21.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.574+0000 7f3151c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f314c1461d0 0x7f314c0d1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:21.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.574+0000 7f3151c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314c0cd560 con 0x7f314c0b4fe0 2024-01-16T15:00:21.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.574+0000 7f3151c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314c0cd840 con 0x7f314c0b3890 2024-01-16T15:00:21.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.574+0000 7f3151c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f314c0cdb20 con 0x7f314c1461d0 2024-01-16T15:00:21.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.574+0000 7f3143e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f314c0b4fe0 0x7f314c0ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:21.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.574+0000 7f3143e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f314c0b4fe0 0x7f314c0ce600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45342/0 (socket says 172.21.15.143:45342) 2024-01-16T15:00:21.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.574+0000 7f3143e00640 1 -- 172.21.15.143:0/2017187224 learned_addr learned my addr 172.21.15.143:0/2017187224 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:21.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.574+0000 7f314aa00640 1 --2- 172.21.15.143:0/2017187224 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f314c0b3890 0x7f314c15c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:21.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.574+0000 7f314b400640 1 --2- 172.21.15.143:0/2017187224 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f314c1461d0 0x7f314c0d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:21.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.575+0000 7f3143e00640 1 -- 172.21.15.143:0/2017187224 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f314c1461d0 msgr2=0x7f314c0d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:21.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.575+0000 7f3143e00640 1 --2- 172.21.15.143:0/2017187224 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f314c1461d0 0x7f314c0d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:21.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.575+0000 7f3143e00640 1 -- 172.21.15.143:0/2017187224 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f314c0b3890 msgr2=0x7f314c15c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:21.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.575+0000 7f3143e00640 1 --2- 172.21.15.143:0/2017187224 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f314c0b3890 0x7f314c15c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:21.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.575+0000 7f3143e00640 1 -- 172.21.15.143:0/2017187224 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f313c009400 con 0x7f314c0b4fe0 2024-01-16T15:00:21.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.575+0000 7f314aa00640 1 --2- 172.21.15.143:0/2017187224 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f314c0b3890 0x7f314c15c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T15:00:21.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.575+0000 7f3143e00640 1 --2- 172.21.15.143:0/2017187224 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f314c0b4fe0 0x7f314c0ce600 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3134000c00 tx=0x7f3134004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:21.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.575+0000 7f3142a00640 1 -- 172.21.15.143:0/2017187224 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3134039c90 con 0x7f314c0b4fe0 2024-01-16T15:00:21.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.575+0000 7f3142a00640 1 -- 172.21.15.143:0/2017187224 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3134039df0 con 0x7f314c0b4fe0 2024-01-16T15:00:21.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.575+0000 7f3151c00640 1 -- 172.21.15.143:0/2017187224 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31340094d0 con 0x7f314c0b4fe0 2024-01-16T15:00:21.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.575+0000 7f3142a00640 1 -- 172.21.15.143:0/2017187224 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3134043640 con 0x7f314c0b4fe0 2024-01-16T15:00:21.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.575+0000 7f3151c00640 1 -- 172.21.15.143:0/2017187224 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f314c0ce0e0 con 0x7f314c0b4fe0 2024-01-16T15:00:21.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.577+0000 7f3151c00640 1 -- 172.21.15.143:0/2017187224 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f314c15d9e0 con 0x7f314c0b4fe0 2024-01-16T15:00:21.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.578+0000 7f3142a00640 1 -- 172.21.15.143:0/2017187224 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f31340437a0 con 0x7f314c0b4fe0 2024-01-16T15:00:21.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.579+0000 7f3142a00640 1 --2- 172.21.15.143:0/2017187224 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f312406b2c0 0x7f312406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:21.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.579+0000 7f3142a00640 1 -- 172.21.15.143:0/2017187224 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f313403e070 con 0x7f314c0b4fe0 2024-01-16T15:00:21.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.579+0000 7f314aa00640 1 --2- 172.21.15.143:0/2017187224 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f312406b2c0 0x7f312406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:21.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.580+0000 7f314aa00640 1 --2- 172.21.15.143:0/2017187224 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f312406b2c0 0x7f312406d780 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f313c009c80 tx=0x7f313c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:21.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.583+0000 7f3142a00640 1 -- 172.21.15.143:0/2017187224 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3134077a40 con 0x7f314c0b4fe0 2024-01-16T15:00:21.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.817+0000 7f3151c00640 1 -- 172.21.15.143:0/2017187224 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f314c067470 con 0x7f314c0b4fe0 2024-01-16T15:00:21.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.818+0000 7f3142a00640 1 -- 172.21.15.143:0/2017187224 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f313407b060 con 0x7f314c0b4fe0 2024-01-16T15:00:21.799 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-16T15:00:21.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.820+0000 7f3151c00640 1 -- 172.21.15.143:0/2017187224 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f312406b2c0 msgr2=0x7f312406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:21.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.820+0000 7f3151c00640 1 --2- 172.21.15.143:0/2017187224 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f312406b2c0 0x7f312406d780 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f313c009c80 tx=0x7f313c008040 comp rx=0 tx=0).stop 2024-01-16T15:00:21.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.820+0000 7f3151c00640 1 -- 172.21.15.143:0/2017187224 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f314c0b4fe0 msgr2=0x7f314c0ce600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:21.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.820+0000 7f3151c00640 1 --2- 172.21.15.143:0/2017187224 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f314c0b4fe0 0x7f314c0ce600 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3134000c00 tx=0x7f3134004730 comp rx=0 tx=0).stop 2024-01-16T15:00:21.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.820+0000 7f3151c00640 1 -- 172.21.15.143:0/2017187224 shutdown_connections 2024-01-16T15:00:21.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.820+0000 7f3151c00640 1 --2- 172.21.15.143:0/2017187224 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f312406b2c0 0x7f312406d780 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:21.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.820+0000 7f3151c00640 1 --2- 172.21.15.143:0/2017187224 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f314c1461d0 0x7f314c0d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:21.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.820+0000 7f3151c00640 1 --2- 172.21.15.143:0/2017187224 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f314c0b4fe0 0x7f314c0ce600 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:21.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.820+0000 7f3151c00640 1 --2- 172.21.15.143:0/2017187224 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f314c0b3890 0x7f314c15c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:21.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.820+0000 7f3151c00640 1 -- 172.21.15.143:0/2017187224 >> 172.21.15.143:0/2017187224 conn(0x7f314c061340 msgr2=0x7f314c15ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:21.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.821+0000 7f3151c00640 1 -- 172.21.15.143:0/2017187224 shutdown_connections 2024-01-16T15:00:21.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:21.821+0000 7f3151c00640 1 -- 172.21.15.143:0/2017187224 wait complete. 2024-01-16T15:00:21.821 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-16T15:00:21.821 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-16T15:00:21.821 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi143.front.sepia.ceph.com 2024-01-16T15:00:21.821 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-16T15:00:21.821 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-16T15:00:21.821 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-16T15:00:21.821 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-16T15:00:21.821 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-16T15:00:21.821 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-16T15:00:21.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:21.822 DEBUG:teuthology.orchestra.run.smithi143:> ip addr 2024-01-16T15:00:21.870 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-16T15:00:21.870 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-16T15:00:21.870 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2024-01-16T15:00:21.870 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:00:21.870 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2024-01-16T15:00:21.870 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:00:21.870 INFO:teuthology.orchestra.run.smithi143.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-16T15:00:21.871 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:94 brd ff:ff:ff:ff:ff:ff 2024-01-16T15:00:21.871 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f0 2024-01-16T15:00:21.871 INFO:teuthology.orchestra.run.smithi143.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-16T15:00:21.871 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:95 brd ff:ff:ff:ff:ff:ff 2024-01-16T15:00:21.871 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f1 2024-01-16T15:00:21.871 INFO:teuthology.orchestra.run.smithi143.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-16T15:00:21.871 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:82 brd ff:ff:ff:ff:ff:ff 2024-01-16T15:00:21.871 INFO:teuthology.orchestra.run.smithi143.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-16T15:00:21.871 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:83 brd ff:ff:ff:ff:ff:ff 2024-01-16T15:00:21.871 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.21.15.143/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-16T15:00:21.871 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft 42595sec preferred_lft 42595sec 2024-01-16T15:00:21.871 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::ec4:7aff:fe88:7783/64 scope link 2024-01-16T15:00:21.871 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:00:21.872 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-01-16T15:00:21.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:21.873 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2024-01-16T15:00:21.873 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-01-16T15:00:21.873 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link add name ceph-brx type bridge 2024-01-16T15:00:21.873 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip addr flush dev ceph-brx 2024-01-16T15:00:21.873 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set ceph-brx up 2024-01-16T15:00:21.873 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-01-16T15:00:21.873 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-01-16T15:00:22.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:22.316 DEBUG:teuthology.orchestra.run.smithi143:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-16T15:00:22.398 INFO:teuthology.orchestra.run.smithi143.stdout:1 2024-01-16T15:00:22.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:22.402 DEBUG:teuthology.orchestra.run.smithi143:> route 2024-01-16T15:00:22.454 INFO:teuthology.orchestra.run.smithi143.stdout:Kernel IP routing table 2024-01-16T15:00:22.454 INFO:teuthology.orchestra.run.smithi143.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-16T15:00:22.454 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-16T15:00:22.454 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-16T15:00:22.454 INFO:teuthology.orchestra.run.smithi143.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-16T15:00:22.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:22.455 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2024-01-16T15:00:22.455 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-01-16T15:00:22.455 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-16T15:00:22.455 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-16T15:00:22.455 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-16T15:00:22.455 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-01-16T15:00:22.983 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:00:22.991 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:00:23.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:23.353 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2024-01-16T15:00:23.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:23.406 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list-id 2024-01-16T15:00:23.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:23.453 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2024-01-16T15:00:23.453 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-01-16T15:00:23.453 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-16T15:00:23.453 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-01-16T15:00:23.454 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-01-16T15:00:23.623 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-01-16T15:00:23.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:23.624 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2024-01-16T15:00:23.624 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-01-16T15:00:23.624 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-01-16T15:00:23.624 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-01-16T15:00:23.624 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-01-16T15:00:23.624 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-01-16T15:00:23.624 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-01-16T15:00:23.624 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-01-16T15:00:23.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:23.881 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2024-01-16T15:00:23.881 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-01-16T15:00:23.881 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set brx.0 up 2024-01-16T15:00:23.881 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set dev brx.0 master ceph-brx 2024-01-16T15:00:23.882 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-01-16T15:00:24.025 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-16T15:00:24.025 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-16T15:00:24.069 INFO:teuthology.orchestra.run.smithi143.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-16T15:00:24.069 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-16T15:00:24.069 DEBUG:teuthology.orchestra.run.smithi143:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-16T15:00:24.156 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-16T15:00:24.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:24.156 DEBUG:teuthology.orchestra.run.smithi143:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-16T15:00:24.340 INFO:teuthology.orchestra.run.smithi143.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-16T15:00:24.340 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-16T15:00:24.340 INFO:teuthology.orchestra.run.smithi143.stdout:invalid new device string format 2024-01-16T15:00:24.951 INFO:teuthology.orchestra.run.smithi143.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-16T15:00:24.951 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-16T15:00:24.951 INFO:teuthology.orchestra.run.smithi143.stdout:invalid new device string format 2024-01-16T15:00:24.952 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: resolved to: "172.21.15.143:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-01-16T15:00:24.952 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-16T15:00:24.992 INFO:teuthology.orchestra.run.smithi143.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-16T15:00:24.992 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-16T15:00:24.992 INFO:teuthology.orchestra.run.smithi143.stdout:invalid new device string format 2024-01-16T15:00:24.992 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: resolved to: "172.21.15.143:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-01-16T15:00:24.992 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-16T15:00:24.992 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: resolved to: "172.21.15.143:6789,172.21.15.204:6789,172.21.15.204:6790" 2024-01-16T15:00:24.992 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.143:6789,172.21.15.204:6789,172.21.15.204:6790:/ 2024-01-16T15:00:24.993 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0" will pass to kernel 2024-01-16T15:00:24.997 INFO:teuthology.orchestra.run.smithi143.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-01-16T15:00:24.997 INFO:teuthology.orchestra.run.smithi143.stdout: You just mounted a file system that supports labels which does not 2024-01-16T15:00:24.997 INFO:teuthology.orchestra.run.smithi143.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-01-16T15:00:24.997 INFO:teuthology.orchestra.run.smithi143.stdout: applications will generate AVC messages and not be allowed access to 2024-01-16T15:00:24.997 INFO:teuthology.orchestra.run.smithi143.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-01-16T15:00:25.003 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-16T15:00:25.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:25.003 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-16T15:00:25.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:25.093 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-16T15:00:25.158 INFO:teuthology.orchestra.run.smithi143.stdout:client4430 2024-01-16T15:00:25.158 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T15:00:25.158 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T15:00:25.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:25.212 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-16T15:00:25.272 INFO:teuthology.orchestra.run.smithi143.stdout:client4430 2024-01-16T15:00:25.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:25.272 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-16T15:00:25.333 INFO:teuthology.orchestra.run.smithi143.stdout:client4430 2024-01-16T15:00:25.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:25.333 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0.client4430/status') 2024-01-16T15:00:25.420 INFO:teuthology.orchestra.run.smithi143.stdout:instance: client.4430 (3)172.21.15.143:0/1508036900 2024-01-16T15:00:25.420 INFO:teuthology.orchestra.run.smithi143.stdout:blocklisted: false 2024-01-16T15:00:25.420 INFO:teuthology.orchestra.run.smithi143.stderr:0+1 records in 2024-01-16T15:00:25.421 INFO:teuthology.orchestra.run.smithi143.stderr:0+1 records out 2024-01-16T15:00:25.421 INFO:teuthology.orchestra.run.smithi143.stderr:71 bytes copied, 6.4877e-05 s, 1.1 MB/s 2024-01-16T15:00:25.424 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-01-16T15:00:25.425 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-01-16T15:00:25.425 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-01-16T15:00:25.425 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-16T15:00:25.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.647+0000 7f05a0200640 1 Processor -- start 2024-01-16T15:00:25.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.647+0000 7f05a0200640 1 -- start start 2024-01-16T15:00:25.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.648+0000 7f05a0200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f059815c7f0 0x7f059806db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:25.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.648+0000 7f05a0200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f05980ce6c0 0x7f05980cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:25.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.648+0000 7f05a0200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05980cece0 con 0x7f05980cf450 2024-01-16T15:00:25.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.648+0000 7f05a0200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05980cfe40 con 0x7f05980ce6c0 2024-01-16T15:00:25.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.648+0000 7f05a0200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05980ceaf0 con 0x7f059815c7f0 2024-01-16T15:00:25.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.649+0000 7f059e200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f05980ce6c0 0x7f05980cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:25.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.649+0000 7f059e200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f05980ce6c0 0x7f05980cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35214/0 (socket says 172.21.15.143:35214) 2024-01-16T15:00:25.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.649+0000 7f059e200640 1 -- 172.21.15.143:0/2496532103 learned_addr learned my addr 172.21.15.143:0/2496532103 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:25.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.649+0000 7f059e200640 1 -- 172.21.15.143:0/2496532103 >> v2:172.21.15.204:3300/0 conn(0x7f059815c7f0 msgr2=0x7f059806db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:25.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.649+0000 7f059e200640 1 --2- 172.21.15.143:0/2496532103 >> v2:172.21.15.204:3300/0 conn(0x7f059815c7f0 0x7f059806db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:25.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.649+0000 7f059e200640 1 -- 172.21.15.143:0/2496532103 >> v1:172.21.15.204:6789/0 conn(0x7f05980cf450 legacy=0x7f05980cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:25.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.649+0000 7f059e200640 1 -- 172.21.15.143:0/2496532103 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059806d040 con 0x7f05980ce6c0 2024-01-16T15:00:25.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.650+0000 7f059e200640 1 --2- 172.21.15.143:0/2496532103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f05980ce6c0 0x7f05980cf900 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f05980d0130 tx=0x7f059002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c976d3c80abbeef4 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:25.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.651+0000 7f0597e00640 1 -- 172.21.15.143:0/2496532103 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059002ca80 con 0x7f05980ce6c0 2024-01-16T15:00:25.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.651+0000 7f0597e00640 1 -- 172.21.15.143:0/2496532103 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059002cbe0 con 0x7f05980ce6c0 2024-01-16T15:00:25.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.651+0000 7f0597e00640 1 -- 172.21.15.143:0/2496532103 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05900343f0 con 0x7f05980ce6c0 2024-01-16T15:00:25.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.651+0000 7f05a0200640 1 -- 172.21.15.143:0/2496532103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f05980ce6c0 msgr2=0x7f05980cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:25.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.651+0000 7f05a0200640 1 --2- 172.21.15.143:0/2496532103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f05980ce6c0 0x7f05980cf900 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f05980d0130 tx=0x7f059002bee0 comp rx=0 tx=0).stop 2024-01-16T15:00:25.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.651+0000 7f05a0200640 1 -- 172.21.15.143:0/2496532103 shutdown_connections 2024-01-16T15:00:25.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.651+0000 7f05a0200640 1 --2- 172.21.15.143:0/2496532103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f05980ce6c0 0x7f05980cf900 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:25.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.651+0000 7f05a0200640 1 --2- 172.21.15.143:0/2496532103 >> v2:172.21.15.204:3300/0 conn(0x7f059815c7f0 0x7f059806db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:25.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.651+0000 7f05a0200640 1 -- 172.21.15.143:0/2496532103 >> 172.21.15.143:0/2496532103 conn(0x7f059806a920 msgr2=0x7f059806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:25.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.651+0000 7f05a0200640 1 -- 172.21.15.143:0/2496532103 shutdown_connections 2024-01-16T15:00:25.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.651+0000 7f05a0200640 1 -- 172.21.15.143:0/2496532103 wait complete. 2024-01-16T15:00:25.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.653+0000 7f05a0200640 1 Processor -- start 2024-01-16T15:00:25.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.654+0000 7f05a0200640 1 -- start start 2024-01-16T15:00:25.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.654+0000 7f05a0200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f05980ce6c0 0x7f0598146910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:25.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.655+0000 7f05a0200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05980cf450 0x7f05980af650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:25.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.655+0000 7f05a0200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f059815c7f0 0x7f05980984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:25.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.655+0000 7f05a0200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05981564a0 con 0x7f059815c7f0 2024-01-16T15:00:25.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.655+0000 7f05a0200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0598146e50 con 0x7f05980cf450 2024-01-16T15:00:25.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.655+0000 7f05a0200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05981f4690 con 0x7f05980ce6c0 2024-01-16T15:00:25.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.655+0000 7f059e200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f059815c7f0 0x7f05980984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:25.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.655+0000 7f059e200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f059815c7f0 0x7f05980984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35226/0 (socket says 172.21.15.143:35226) 2024-01-16T15:00:25.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.655+0000 7f059e200640 1 -- 172.21.15.143:0/3272608202 learned_addr learned my addr 172.21.15.143:0/3272608202 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:25.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.655+0000 7f059d800640 1 --2- 172.21.15.143:0/3272608202 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f05980ce6c0 0x7f0598146910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:25.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.655+0000 7f059ce00640 1 --2- 172.21.15.143:0/3272608202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05980cf450 0x7f05980af650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:25.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.655+0000 7f059e200640 1 -- 172.21.15.143:0/3272608202 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f05980ce6c0 msgr2=0x7f0598146910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:25.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.655+0000 7f059e200640 1 --2- 172.21.15.143:0/3272608202 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f05980ce6c0 0x7f0598146910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:25.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.655+0000 7f059e200640 1 -- 172.21.15.143:0/3272608202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05980cf450 msgr2=0x7f05980af650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:25.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.655+0000 7f059e200640 1 --2- 172.21.15.143:0/3272608202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05980cf450 0x7f05980af650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:25.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.655+0000 7f059e200640 1 -- 172.21.15.143:0/3272608202 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0590009b30 con 0x7f059815c7f0 2024-01-16T15:00:25.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.655+0000 7f059ce00640 1 --2- 172.21.15.143:0/3272608202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05980cf450 0x7f05980af650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:25.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.656+0000 7f059e200640 1 --2- 172.21.15.143:0/3272608202 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f059815c7f0 0x7f05980984f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0590036040 tx=0x7f0590004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:25.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.656+0000 7f0595600640 1 -- 172.21.15.143:0/3272608202 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0590009510 con 0x7f059815c7f0 2024-01-16T15:00:25.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.656+0000 7f0595600640 1 -- 172.21.15.143:0/3272608202 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059003e450 con 0x7f059815c7f0 2024-01-16T15:00:25.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.656+0000 7f0595600640 1 -- 172.21.15.143:0/3272608202 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059003e7e0 con 0x7f059815c7f0 2024-01-16T15:00:25.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.656+0000 7f05a0200640 1 -- 172.21.15.143:0/3272608202 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05981561c0 con 0x7f059815c7f0 2024-01-16T15:00:25.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.656+0000 7f05a0200640 1 -- 172.21.15.143:0/3272608202 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05981f4ea0 con 0x7f059815c7f0 2024-01-16T15:00:25.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.657+0000 7f05a0200640 1 -- 172.21.15.143:0/3272608202 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05980ae6a0 con 0x7f059815c7f0 2024-01-16T15:00:25.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.658+0000 7f0595600640 1 -- 172.21.15.143:0/3272608202 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f059002cd50 con 0x7f059815c7f0 2024-01-16T15:00:25.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.659+0000 7f0595600640 1 --2- 172.21.15.143:0/3272608202 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f057006b2c0 0x7f057006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:25.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.659+0000 7f0595600640 1 -- 172.21.15.143:0/3272608202 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05900ac750 con 0x7f059815c7f0 2024-01-16T15:00:25.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.659+0000 7f059d800640 1 --2- 172.21.15.143:0/3272608202 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f057006b2c0 0x7f057006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:25.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.660+0000 7f059d800640 1 --2- 172.21.15.143:0/3272608202 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f057006b2c0 0x7f057006d780 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f058800a770 tx=0x7f0588008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:25.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.664+0000 7f0595600640 1 -- 172.21.15.143:0/3272608202 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f05900775c0 con 0x7f059815c7f0 2024-01-16T15:00:25.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.897+0000 7f05a0200640 1 -- 172.21.15.143:0/3272608202 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0598067470 con 0x7f059815c7f0 2024-01-16T15:00:25.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.898+0000 7f0595600640 1 -- 172.21.15.143:0/3272608202 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f059007abe0 con 0x7f059815c7f0 2024-01-16T15:00:25.879 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-16T15:00:25.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.900+0000 7f05a0200640 1 -- 172.21.15.143:0/3272608202 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f057006b2c0 msgr2=0x7f057006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:25.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.900+0000 7f05a0200640 1 --2- 172.21.15.143:0/3272608202 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f057006b2c0 0x7f057006d780 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f058800a770 tx=0x7f0588008040 comp rx=0 tx=0).stop 2024-01-16T15:00:25.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.900+0000 7f05a0200640 1 -- 172.21.15.143:0/3272608202 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f059815c7f0 msgr2=0x7f05980984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:25.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.900+0000 7f05a0200640 1 --2- 172.21.15.143:0/3272608202 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f059815c7f0 0x7f05980984f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0590036040 tx=0x7f0590004a40 comp rx=0 tx=0).stop 2024-01-16T15:00:25.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.901+0000 7f05a0200640 1 -- 172.21.15.143:0/3272608202 shutdown_connections 2024-01-16T15:00:25.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.901+0000 7f05a0200640 1 --2- 172.21.15.143:0/3272608202 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f057006b2c0 0x7f057006d780 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:25.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.901+0000 7f05a0200640 1 --2- 172.21.15.143:0/3272608202 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f059815c7f0 0x7f05980984f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:25.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.901+0000 7f05a0200640 1 --2- 172.21.15.143:0/3272608202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05980cf450 0x7f05980af650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:25.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.901+0000 7f05a0200640 1 --2- 172.21.15.143:0/3272608202 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f05980ce6c0 0x7f0598146910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:25.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.901+0000 7f05a0200640 1 -- 172.21.15.143:0/3272608202 >> 172.21.15.143:0/3272608202 conn(0x7f059806a920 msgr2=0x7f05980cdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:25.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.901+0000 7f05a0200640 1 -- 172.21.15.143:0/3272608202 shutdown_connections 2024-01-16T15:00:25.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:25.901+0000 7f05a0200640 1 -- 172.21.15.143:0/3272608202 wait complete. 2024-01-16T15:00:25.900 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-16T15:00:25.900 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-16T15:00:25.900 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi143.front.sepia.ceph.com 2024-01-16T15:00:25.901 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-01-16T15:00:25.901 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-01-16T15:00:25.901 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-16T15:00:25.901 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-16T15:00:25.901 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-16T15:00:25.901 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-01-16T15:00:25.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:25.901 DEBUG:teuthology.orchestra.run.smithi143:> ip addr 2024-01-16T15:00:25.953 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-16T15:00:25.953 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-16T15:00:25.953 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2024-01-16T15:00:25.954 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:00:25.954 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2024-01-16T15:00:25.954 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:00:25.954 INFO:teuthology.orchestra.run.smithi143.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-16T15:00:25.954 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:94 brd ff:ff:ff:ff:ff:ff 2024-01-16T15:00:25.954 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f0 2024-01-16T15:00:25.954 INFO:teuthology.orchestra.run.smithi143.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-16T15:00:25.955 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:95 brd ff:ff:ff:ff:ff:ff 2024-01-16T15:00:25.955 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f1 2024-01-16T15:00:25.955 INFO:teuthology.orchestra.run.smithi143.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-16T15:00:25.955 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:82 brd ff:ff:ff:ff:ff:ff 2024-01-16T15:00:25.955 INFO:teuthology.orchestra.run.smithi143.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-16T15:00:25.955 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:83 brd ff:ff:ff:ff:ff:ff 2024-01-16T15:00:25.955 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.21.15.143/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-16T15:00:25.955 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft 42591sec preferred_lft 42591sec 2024-01-16T15:00:25.956 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::ec4:7aff:fe88:7783/64 scope link 2024-01-16T15:00:25.956 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:00:25.956 INFO:teuthology.orchestra.run.smithi143.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-16T15:00:25.956 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether ce:00:1c:52:30:ff brd ff:ff:ff:ff:ff:ff 2024-01-16T15:00:25.956 INFO:teuthology.orchestra.run.smithi143.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-16T15:00:25.956 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:00:25.956 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::d08e:67ff:fe88:3a91/64 scope link 2024-01-16T15:00:25.956 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:00:25.956 INFO:teuthology.orchestra.run.smithi143.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-16T15:00:25.956 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether ce:00:1c:52:30:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-16T15:00:25.957 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::cc00:1cff:fe52:30ff/64 scope link 2024-01-16T15:00:25.957 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:00:25.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:25.958 DEBUG:teuthology.orchestra.run.smithi143:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-16T15:00:26.044 INFO:teuthology.orchestra.run.smithi143.stdout:1 2024-01-16T15:00:26.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:26.049 DEBUG:teuthology.orchestra.run.smithi143:> route 2024-01-16T15:00:26.100 INFO:teuthology.orchestra.run.smithi143.stdout:Kernel IP routing table 2024-01-16T15:00:26.100 INFO:teuthology.orchestra.run.smithi143.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-16T15:00:26.100 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-16T15:00:26.100 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-16T15:00:26.100 INFO:teuthology.orchestra.run.smithi143.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-16T15:00:26.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:26.101 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2024-01-16T15:00:26.101 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-01-16T15:00:26.101 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-16T15:00:26.101 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-16T15:00:26.101 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-16T15:00:26.101 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-01-16T15:00:26.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:26.285 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2024-01-16T15:00:26.336 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-16T15:00:26.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:26.337 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list-id 2024-01-16T15:00:26.386 INFO:teuthology.orchestra.run.smithi143.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-01-16T15:00:26.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:26.387 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2024-01-16T15:00:26.388 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-01-16T15:00:26.388 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-16T15:00:26.388 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-01-16T15:00:26.388 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-01-16T15:00:26.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:26.520 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-01-16T15:00:26.608 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-16T15:00:26.609 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-16T15:00:26.609 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2024-01-16T15:00:26.609 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:00:26.609 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2024-01-16T15:00:26.609 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:00:26.609 INFO:teuthology.orchestra.run.smithi143.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-16T15:00:26.609 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether ba:94:f5:f5:e2:d2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-01-16T15:00:26.609 INFO:teuthology.orchestra.run.smithi143.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-01-16T15:00:26.609 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:00:26.609 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::b894:f5ff:fef5:e2d2/64 scope link 2024-01-16T15:00:26.609 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:00:26.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:26.613 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-01-16T15:00:26.698 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-16T15:00:26.699 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-16T15:00:26.699 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2024-01-16T15:00:26.699 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:00:26.699 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2024-01-16T15:00:26.699 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:00:26.699 INFO:teuthology.orchestra.run.smithi143.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-16T15:00:26.699 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether ba:94:f5:f5:e2:d2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-01-16T15:00:26.699 INFO:teuthology.orchestra.run.smithi143.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-01-16T15:00:26.699 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:00:26.699 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::b894:f5ff:fef5:e2d2/64 scope link 2024-01-16T15:00:26.699 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:00:26.703 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-01-16T15:00:26.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:26.703 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2024-01-16T15:00:26.703 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-01-16T15:00:26.704 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-01-16T15:00:26.704 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-01-16T15:00:26.704 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-01-16T15:00:26.704 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-01-16T15:00:26.704 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-01-16T15:00:26.704 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-01-16T15:00:26.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:26.964 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2024-01-16T15:00:26.964 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-01-16T15:00:26.964 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set brx.1 up 2024-01-16T15:00:26.964 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set dev brx.1 master ceph-brx 2024-01-16T15:00:26.964 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-01-16T15:00:27.124 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-16T15:00:27.125 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-01-16T15:00:27.166 INFO:teuthology.orchestra.run.smithi143.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-01-16T15:00:27.166 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-16T15:00:27.167 DEBUG:teuthology.orchestra.run.smithi143:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-01-16T15:00:27.257 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-16T15:00:27.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:27.258 DEBUG:teuthology.orchestra.run.smithi143:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-01-16T15:00:27.392 INFO:teuthology.orchestra.run.smithi143.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-01-16T15:00:27.392 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-01-16T15:00:27.392 INFO:teuthology.orchestra.run.smithi143.stdout:invalid new device string format 2024-01-16T15:00:27.414 INFO:teuthology.orchestra.run.smithi143.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-01-16T15:00:27.414 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-01-16T15:00:27.414 INFO:teuthology.orchestra.run.smithi143.stdout:invalid new device string format 2024-01-16T15:00:27.414 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: resolved to: "172.21.15.143:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-01-16T15:00:27.414 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-16T15:00:27.440 INFO:teuthology.orchestra.run.smithi143.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-01-16T15:00:27.440 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-01-16T15:00:27.440 INFO:teuthology.orchestra.run.smithi143.stdout:invalid new device string format 2024-01-16T15:00:27.440 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: resolved to: "172.21.15.143:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-01-16T15:00:27.441 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-16T15:00:27.441 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: resolved to: "172.21.15.143:6789,172.21.15.204:6789,172.21.15.204:6790" 2024-01-16T15:00:27.441 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.143:6789,172.21.15.204:6789,172.21.15.204:6790:/ 2024-01-16T15:00:27.441 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0" will pass to kernel 2024-01-16T15:00:27.442 INFO:teuthology.orchestra.run.smithi143.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-01-16T15:00:27.442 INFO:teuthology.orchestra.run.smithi143.stdout: You just mounted a file system that supports labels which does not 2024-01-16T15:00:27.442 INFO:teuthology.orchestra.run.smithi143.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-01-16T15:00:27.443 INFO:teuthology.orchestra.run.smithi143.stdout: applications will generate AVC messages and not be allowed access to 2024-01-16T15:00:27.443 INFO:teuthology.orchestra.run.smithi143.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-01-16T15:00:27.446 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-16T15:00:27.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:27.447 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-01-16T15:00:27.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:27.530 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-16T15:00:27.584 INFO:teuthology.orchestra.run.smithi143.stdout:client4686 2024-01-16T15:00:27.585 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T15:00:27.585 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T15:00:27.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:27.635 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-16T15:00:27.737 INFO:teuthology.orchestra.run.smithi143.stdout:client4686 2024-01-16T15:00:27.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:27.737 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-16T15:00:27.794 INFO:teuthology.orchestra.run.smithi143.stdout:client4686 2024-01-16T15:00:27.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:27.795 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0.client4686/status') 2024-01-16T15:00:27.880 INFO:teuthology.orchestra.run.smithi143.stdout:instance: client.4686 (3)192.168.0.2:0/3024111376 2024-01-16T15:00:27.880 INFO:teuthology.orchestra.run.smithi143.stdout:blocklisted: false 2024-01-16T15:00:27.881 INFO:teuthology.orchestra.run.smithi143.stderr:0+1 records in 2024-01-16T15:00:27.881 INFO:teuthology.orchestra.run.smithi143.stderr:0+1 records out 2024-01-16T15:00:27.881 INFO:teuthology.orchestra.run.smithi143.stderr:69 bytes copied, 6.2444e-05 s, 1.1 MB/s 2024-01-16T15:00:27.885 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-01-16T15:00:27.885 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-01-16T15:00:27.885 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-16T15:00:27.885 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-16T15:00:28.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.113+0000 7f71cf400640 1 Processor -- start 2024-01-16T15:00:28.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.113+0000 7f71cf400640 1 -- start start 2024-01-16T15:00:28.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.115+0000 7f71cf400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71c80d1740 0x7f71c80ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:28.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.115+0000 7f71cf400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f71c80d0f40 0x7f71c80d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:28.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.115+0000 7f71cf400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c80cecc0 con 0x7f71c80cf790 2024-01-16T15:00:28.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.115+0000 7f71cf400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71c80d2060 con 0x7f71c80d0f40 2024-01-16T15:00:28.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.115+0000 7f71cf400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c80cef60 con 0x7f71c80d1740 2024-01-16T15:00:28.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.115+0000 7f71cea00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f71c80cf790 0x7f71c80cfc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48526/0 (socket says 172.21.15.143:48526) 2024-01-16T15:00:28.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.115+0000 7f71cea00640 1 -- 172.21.15.143:0/3079584934 learned_addr learned my addr 172.21.15.143:0/3079584934 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:28.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.115+0000 7f71cd600640 1 --2- 172.21.15.143:0/3079584934 >> v2:172.21.15.204:3300/0 conn(0x7f71c80d0f40 0x7f71c80d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:28.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.116+0000 7f71ce000640 1 --2- 172.21.15.143:0/3079584934 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71c80d1740 0x7f71c80ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:28.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.116+0000 7f71ccc00640 1 -- 172.21.15.143:0/3079584934 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1314202904 0 0) 0x7f71c80cecc0 con 0x7f71c80cf790 2024-01-16T15:00:28.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.116+0000 7f71ccc00640 1 -- 172.21.15.143:0/3079584934 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71a8003520 con 0x7f71c80cf790 2024-01-16T15:00:28.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.116+0000 7f71ccc00640 1 -- 172.21.15.143:0/3079584934 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1582941821 0 0) 0x7f71a8003520 con 0x7f71c80cf790 2024-01-16T15:00:28.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.116+0000 7f71ccc00640 1 -- 172.21.15.143:0/3079584934 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71c80d1740 msgr2=0x7f71c80ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:28.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.116+0000 7f71ccc00640 1 --2- 172.21.15.143:0/3079584934 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71c80d1740 0x7f71c80ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:28.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.116+0000 7f71ccc00640 1 -- 172.21.15.143:0/3079584934 >> v2:172.21.15.204:3300/0 conn(0x7f71c80d0f40 msgr2=0x7f71c80d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:28.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.116+0000 7f71ccc00640 1 --2- 172.21.15.143:0/3079584934 >> v2:172.21.15.204:3300/0 conn(0x7f71c80d0f40 0x7f71c80d1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:28.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.116+0000 7f71ccc00640 1 -- 172.21.15.143:0/3079584934 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c80b2ac0 con 0x7f71c80cf790 2024-01-16T15:00:28.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.116+0000 7f71ccc00640 1 -- 172.21.15.143:0/3079584934 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7f71b8001670 con 0x7f71c80cf790 2024-01-16T15:00:28.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.116+0000 7f71ccc00640 1 -- 172.21.15.143:0/3079584934 >> v1:172.21.15.143:6789/0 conn(0x7f71c80cf790 legacy=0x7f71c80cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:28.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.116+0000 7f71ccc00640 1 --2- 172.21.15.143:0/3079584934 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71a8003970 0x7f71a8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:28.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.116+0000 7f71ccc00640 1 --2- 172.21.15.143:0/3079584934 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71c80d1740 0x7f71a80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:28.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.116+0000 7f71ccc00640 1 --2- 172.21.15.143:0/3079584934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71a8004d00 0x7f71a8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:28.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.116+0000 7f71ccc00640 1 -- 172.21.15.143:0/3079584934 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c80d2060 con 0x7f71a8003970 2024-01-16T15:00:28.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.116+0000 7f71ccc00640 1 -- 172.21.15.143:0/3079584934 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c80cef60 con 0x7f71a8004d00 2024-01-16T15:00:28.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.116+0000 7f71ccc00640 1 -- 172.21.15.143:0/3079584934 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a800b9b0 con 0x7f71c80d1740 2024-01-16T15:00:28.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.116+0000 7f71cd600640 1 --2- 172.21.15.143:0/3079584934 >> v2:172.21.15.204:3300/0 conn(0x7f71c80d0f40 0x7f71c80d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T15:00:28.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.117+0000 7f71ce000640 1 --2- 172.21.15.143:0/3079584934 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71a8003970 0x7f71a8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:28.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.117+0000 7f71cea00640 1 --2- 172.21.15.143:0/3079584934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71a8004d00 0x7f71a8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:28.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.117+0000 7f71cd600640 1 --2- 172.21.15.143:0/3079584934 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71c80d1740 0x7f71a80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:28.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.117+0000 7f71ce000640 1 -- 172.21.15.143:0/3079584934 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71c80d1740 msgr2=0x7f71a80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:28.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.117+0000 7f71ce000640 1 --2- 172.21.15.143:0/3079584934 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71c80d1740 0x7f71a80047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:28.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.117+0000 7f71ce000640 1 -- 172.21.15.143:0/3079584934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71a8004d00 msgr2=0x7f71a8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:28.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.117+0000 7f71ce000640 1 --2- 172.21.15.143:0/3079584934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71a8004d00 0x7f71a8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:28.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.117+0000 7f71ce000640 1 -- 172.21.15.143:0/3079584934 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a8003520 con 0x7f71a8003970 2024-01-16T15:00:28.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.118+0000 7f71ce000640 1 --2- 172.21.15.143:0/3079584934 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71a8003970 0x7f71a8003d90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f71c0009aa0 tx=0x7f71a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe615bb8da50f38 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:28.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.118+0000 7f71ccc00640 1 -- 172.21.15.143:0/3079584934 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c002c6b0 con 0x7f71a8003970 2024-01-16T15:00:28.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.119+0000 7f71ccc00640 1 -- 172.21.15.143:0/3079584934 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c002c810 con 0x7f71a8003970 2024-01-16T15:00:28.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.119+0000 7f71ccc00640 1 -- 172.21.15.143:0/3079584934 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c002cbc0 con 0x7f71a8003970 2024-01-16T15:00:28.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.119+0000 7f71cf400640 1 -- 172.21.15.143:0/3079584934 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71a8003970 msgr2=0x7f71a8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:28.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.119+0000 7f71cf400640 1 --2- 172.21.15.143:0/3079584934 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71a8003970 0x7f71a8003d90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f71c0009aa0 tx=0x7f71a80057f0 comp rx=0 tx=0).stop 2024-01-16T15:00:28.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.119+0000 7f71cea00640 1 -- 172.21.15.143:0/3079584934 reap_dead start 2024-01-16T15:00:28.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.119+0000 7f71cf400640 1 -- 172.21.15.143:0/3079584934 shutdown_connections 2024-01-16T15:00:28.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.119+0000 7f71cf400640 1 -- 172.21.15.143:0/3079584934 >> 172.21.15.143:0/3079584934 conn(0x7f71c809c500 msgr2=0x7f71c80d3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:28.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.119+0000 7f71cf400640 1 -- 172.21.15.143:0/3079584934 shutdown_connections 2024-01-16T15:00:28.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.119+0000 7f71cf400640 1 -- 172.21.15.143:0/3079584934 wait complete. 2024-01-16T15:00:28.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.121+0000 7f71cf400640 1 Processor -- start 2024-01-16T15:00:28.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.121+0000 7f71cf400640 1 -- start start 2024-01-16T15:00:28.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.122+0000 7f71cf400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71a8003970 0x7f71c8152210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:28.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.122+0000 7f71cf400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71c815e480 0x7f71c8160070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:28.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.122+0000 7f71cf400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71c81605b0 0x7f71c8153480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:28.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.122+0000 7f71cf400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c8152750 con 0x7f71c81605b0 2024-01-16T15:00:28.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.122+0000 7f71cf400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c8160a30 con 0x7f71a8003970 2024-01-16T15:00:28.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.122+0000 7f71cf400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c8156750 con 0x7f71c815e480 2024-01-16T15:00:28.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.122+0000 7f71cea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71c81605b0 0x7f71c8153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:28.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.123+0000 7f71cea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71c81605b0 0x7f71c8153480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35238/0 (socket says 172.21.15.143:35238) 2024-01-16T15:00:28.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.123+0000 7f71cea00640 1 -- 172.21.15.143:0/1655343733 learned_addr learned my addr 172.21.15.143:0/1655343733 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:28.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.123+0000 7f71ce000640 1 --2- 172.21.15.143:0/1655343733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71a8003970 0x7f71c8152210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:28.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.123+0000 7f71cd600640 1 --2- 172.21.15.143:0/1655343733 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71c815e480 0x7f71c8160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:28.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.123+0000 7f71cea00640 1 -- 172.21.15.143:0/1655343733 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71c815e480 msgr2=0x7f71c8160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:28.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.123+0000 7f71cea00640 1 --2- 172.21.15.143:0/1655343733 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71c815e480 0x7f71c8160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:28.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.123+0000 7f71cea00640 1 -- 172.21.15.143:0/1655343733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71a8003970 msgr2=0x7f71c8152210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:28.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.123+0000 7f71cea00640 1 --2- 172.21.15.143:0/1655343733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71a8003970 0x7f71c8152210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:28.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.123+0000 7f71cea00640 1 -- 172.21.15.143:0/1655343733 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c8155b40 con 0x7f71c81605b0 2024-01-16T15:00:28.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.123+0000 7f71cea00640 1 --2- 172.21.15.143:0/1655343733 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71c81605b0 0x7f71c8153480 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f71c808caa0 tx=0x7f71b800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:28.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.123+0000 7f71c6000640 1 -- 172.21.15.143:0/1655343733 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b800eed0 con 0x7f71c81605b0 2024-01-16T15:00:28.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.124+0000 7f71c6000640 1 -- 172.21.15.143:0/1655343733 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b800c610 con 0x7f71c81605b0 2024-01-16T15:00:28.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.124+0000 7f71c6000640 1 -- 172.21.15.143:0/1655343733 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b800c9c0 con 0x7f71c81605b0 2024-01-16T15:00:28.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.124+0000 7f71cf400640 1 -- 172.21.15.143:0/1655343733 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c8155e20 con 0x7f71c81605b0 2024-01-16T15:00:28.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.124+0000 7f71cf400640 1 -- 172.21.15.143:0/1655343733 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c81563e0 con 0x7f71c81605b0 2024-01-16T15:00:28.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.125+0000 7f71cf400640 1 -- 172.21.15.143:0/1655343733 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71c804e840 con 0x7f71c81605b0 2024-01-16T15:00:28.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.126+0000 7f71c6000640 1 -- 172.21.15.143:0/1655343733 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f71b800cb20 con 0x7f71c81605b0 2024-01-16T15:00:28.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.127+0000 7f71c6000640 1 --2- 172.21.15.143:0/1655343733 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f71a006b2c0 0x7f71a006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:28.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.127+0000 7f71c6000640 1 -- 172.21.15.143:0/1655343733 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f71b8089910 con 0x7f71c81605b0 2024-01-16T15:00:28.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.127+0000 7f71ce000640 1 --2- 172.21.15.143:0/1655343733 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f71a006b2c0 0x7f71a006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:28.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.128+0000 7f71ce000640 1 --2- 172.21.15.143:0/1655343733 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f71a006b2c0 0x7f71a006d780 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f71c0009aa0 tx=0x7f71c0030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:28.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.131+0000 7f71c6000640 1 -- 172.21.15.143:0/1655343733 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f71b80546e0 con 0x7f71c81605b0 2024-01-16T15:00:28.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.357+0000 7f71cf400640 1 -- 172.21.15.143:0/1655343733 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f71c804e840 con 0x7f71c81605b0 2024-01-16T15:00:28.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.358+0000 7f71c6000640 1 -- 172.21.15.143:0/1655343733 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f71b8057d00 con 0x7f71c81605b0 2024-01-16T15:00:28.339 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-16T15:00:28.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.360+0000 7f71cf400640 1 -- 172.21.15.143:0/1655343733 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f71a006b2c0 msgr2=0x7f71a006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:28.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.360+0000 7f71cf400640 1 --2- 172.21.15.143:0/1655343733 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f71a006b2c0 0x7f71a006d780 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f71c0009aa0 tx=0x7f71c0030000 comp rx=0 tx=0).stop 2024-01-16T15:00:28.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.360+0000 7f71cf400640 1 -- 172.21.15.143:0/1655343733 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71c81605b0 msgr2=0x7f71c8153480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:28.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.360+0000 7f71cf400640 1 --2- 172.21.15.143:0/1655343733 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71c81605b0 0x7f71c8153480 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f71c808caa0 tx=0x7f71b800eb90 comp rx=0 tx=0).stop 2024-01-16T15:00:28.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.361+0000 7f71cf400640 1 -- 172.21.15.143:0/1655343733 shutdown_connections 2024-01-16T15:00:28.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.361+0000 7f71cf400640 1 --2- 172.21.15.143:0/1655343733 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f71a006b2c0 0x7f71a006d780 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:28.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.361+0000 7f71cf400640 1 --2- 172.21.15.143:0/1655343733 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71c81605b0 0x7f71c8153480 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:28.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.361+0000 7f71cf400640 1 --2- 172.21.15.143:0/1655343733 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71c815e480 0x7f71c8160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:28.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.361+0000 7f71cf400640 1 --2- 172.21.15.143:0/1655343733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71a8003970 0x7f71c8152210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:28.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.361+0000 7f71cf400640 1 -- 172.21.15.143:0/1655343733 >> 172.21.15.143:0/1655343733 conn(0x7f71c809c500 msgr2=0x7f71c80b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:28.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.361+0000 7f71cf400640 1 -- 172.21.15.143:0/1655343733 shutdown_connections 2024-01-16T15:00:28.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:28.361+0000 7f71cf400640 1 -- 172.21.15.143:0/1655343733 wait complete. 2024-01-16T15:00:28.361 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-16T15:00:28.362 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-16T15:00:28.362 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi204.front.sepia.ceph.com 2024-01-16T15:00:28.362 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-16T15:00:28.362 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-16T15:00:28.362 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-16T15:00:28.362 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-16T15:00:28.362 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-16T15:00:28.362 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-16T15:00:28.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:28.362 DEBUG:teuthology.orchestra.run.smithi204:> ip addr 2024-01-16T15:00:28.419 INFO:teuthology.orchestra.run.smithi204.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-16T15:00:28.419 INFO:teuthology.orchestra.run.smithi204.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-16T15:00:28.419 INFO:teuthology.orchestra.run.smithi204.stdout: inet 127.0.0.1/8 scope host lo 2024-01-16T15:00:28.419 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:00:28.419 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 ::1/128 scope host 2024-01-16T15:00:28.419 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:00:28.419 INFO:teuthology.orchestra.run.smithi204.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-16T15:00:28.420 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2a brd ff:ff:ff:ff:ff:ff 2024-01-16T15:00:28.420 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f0 2024-01-16T15:00:28.420 INFO:teuthology.orchestra.run.smithi204.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-16T15:00:28.420 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2b brd ff:ff:ff:ff:ff:ff 2024-01-16T15:00:28.420 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f1 2024-01-16T15:00:28.420 INFO:teuthology.orchestra.run.smithi204.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-16T15:00:28.420 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:aa brd ff:ff:ff:ff:ff:ff 2024-01-16T15:00:28.420 INFO:teuthology.orchestra.run.smithi204.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-16T15:00:28.420 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:ab brd ff:ff:ff:ff:ff:ff 2024-01-16T15:00:28.420 INFO:teuthology.orchestra.run.smithi204.stdout: inet 172.21.15.204/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-16T15:00:28.420 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft 42615sec preferred_lft 42615sec 2024-01-16T15:00:28.420 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::ec4:7aff:fe88:77ab/64 scope link 2024-01-16T15:00:28.420 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:00:28.421 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-01-16T15:00:28.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:28.422 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec bash -c ' 2024-01-16T15:00:28.422 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-01-16T15:00:28.422 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link add name ceph-brx type bridge 2024-01-16T15:00:28.422 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip addr flush dev ceph-brx 2024-01-16T15:00:28.422 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set ceph-brx up 2024-01-16T15:00:28.422 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-01-16T15:00:28.422 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-01-16T15:00:28.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:28.858 DEBUG:teuthology.orchestra.run.smithi204:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-16T15:00:28.941 INFO:teuthology.orchestra.run.smithi204.stdout:1 2024-01-16T15:00:28.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:28.945 DEBUG:teuthology.orchestra.run.smithi204:> route 2024-01-16T15:00:29.001 INFO:teuthology.orchestra.run.smithi204.stdout:Kernel IP routing table 2024-01-16T15:00:29.001 INFO:teuthology.orchestra.run.smithi204.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-16T15:00:29.002 INFO:teuthology.orchestra.run.smithi204.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-16T15:00:29.002 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-16T15:00:29.002 INFO:teuthology.orchestra.run.smithi204.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-16T15:00:29.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:29.003 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec bash -c ' 2024-01-16T15:00:29.003 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-01-16T15:00:29.003 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-16T15:00:29.003 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-16T15:00:29.003 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-16T15:00:29.003 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-01-16T15:00:29.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:29.741 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-01-16T15:00:29.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:29.793 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list-id 2024-01-16T15:00:29.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:29.844 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec bash -c ' 2024-01-16T15:00:29.844 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-01-16T15:00:29.844 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-16T15:00:29.844 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-01-16T15:00:29.844 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-01-16T15:00:29.987 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-01-16T15:00:29.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:29.987 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec bash -c ' 2024-01-16T15:00:29.987 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-01-16T15:00:29.988 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-01-16T15:00:29.988 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-01-16T15:00:29.988 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-01-16T15:00:29.988 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-01-16T15:00:29.988 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-01-16T15:00:29.988 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-01-16T15:00:30.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:30.267 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec bash -c ' 2024-01-16T15:00:30.268 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-01-16T15:00:30.268 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set brx.0 up 2024-01-16T15:00:30.268 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set dev brx.0 master ceph-brx 2024-01-16T15:00:30.268 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-01-16T15:00:30.438 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-16T15:00:30.439 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-16T15:00:30.481 INFO:teuthology.orchestra.run.smithi204.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-16T15:00:30.481 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-16T15:00:30.481 DEBUG:teuthology.orchestra.run.smithi204:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-16T15:00:30.576 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-16T15:00:30.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:30.576 DEBUG:teuthology.orchestra.run.smithi204:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-16T15:00:30.723 INFO:teuthology.orchestra.run.smithi204.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-16T15:00:30.723 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-16T15:00:30.724 INFO:teuthology.orchestra.run.smithi204.stdout:invalid new device string format 2024-01-16T15:00:31.348 INFO:teuthology.orchestra.run.smithi204.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-16T15:00:31.348 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-16T15:00:31.348 INFO:teuthology.orchestra.run.smithi204.stdout:invalid new device string format 2024-01-16T15:00:31.348 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: resolved to: "172.21.15.143:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-01-16T15:00:31.348 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-16T15:00:31.387 INFO:teuthology.orchestra.run.smithi204.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-16T15:00:31.387 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-16T15:00:31.387 INFO:teuthology.orchestra.run.smithi204.stdout:invalid new device string format 2024-01-16T15:00:31.387 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: resolved to: "172.21.15.143:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-01-16T15:00:31.387 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-16T15:00:31.387 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: resolved to: "172.21.15.143:6789,172.21.15.204:6789,172.21.15.204:6790" 2024-01-16T15:00:31.387 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.143:6789,172.21.15.204:6789,172.21.15.204:6790:/ 2024-01-16T15:00:31.387 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0" will pass to kernel 2024-01-16T15:00:31.395 INFO:teuthology.orchestra.run.smithi204.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-01-16T15:00:31.395 INFO:teuthology.orchestra.run.smithi204.stdout: You just mounted a file system that supports labels which does not 2024-01-16T15:00:31.395 INFO:teuthology.orchestra.run.smithi204.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-01-16T15:00:31.395 INFO:teuthology.orchestra.run.smithi204.stdout: applications will generate AVC messages and not be allowed access to 2024-01-16T15:00:31.395 INFO:teuthology.orchestra.run.smithi204.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-01-16T15:00:31.401 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-16T15:00:31.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:31.401 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-16T15:00:31.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:31.492 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-16T15:00:31.551 INFO:teuthology.orchestra.run.smithi204.stdout:client4439 2024-01-16T15:00:31.551 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-16T15:00:31.551 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T15:00:31.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:31.604 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-16T15:00:31.656 INFO:teuthology.orchestra.run.smithi204.stdout:client4439 2024-01-16T15:00:31.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:31.657 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-16T15:00:31.713 INFO:teuthology.orchestra.run.smithi204.stdout:client4439 2024-01-16T15:00:31.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:31.714 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0.client4439/status') 2024-01-16T15:00:31.798 INFO:teuthology.orchestra.run.smithi204.stdout:instance: client.4439 (3)192.168.0.1:0/1196230078 2024-01-16T15:00:31.798 INFO:teuthology.orchestra.run.smithi204.stdout:blocklisted: false 2024-01-16T15:00:31.799 INFO:teuthology.orchestra.run.smithi204.stderr:0+1 records in 2024-01-16T15:00:31.799 INFO:teuthology.orchestra.run.smithi204.stderr:0+1 records out 2024-01-16T15:00:31.799 INFO:teuthology.orchestra.run.smithi204.stderr:69 bytes copied, 6.5245e-05 s, 1.1 MB/s 2024-01-16T15:00:31.803 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-01-16T15:00:31.803 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-01-16T15:00:31.803 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-01-16T15:00:31.803 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-16T15:00:32.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.033+0000 7f92a4200640 1 Processor -- start 2024-01-16T15:00:32.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.033+0000 7f92a4200640 1 -- start start 2024-01-16T15:00:32.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.034+0000 7f92a4200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f929c15df80 0x7f929c0ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:32.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.034+0000 7f92a4200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f929c15ab80 con 0x7f929c0cd470 2024-01-16T15:00:32.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.034+0000 7f92a4200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f929c15e3c0 con 0x7f929c0b2250 2024-01-16T15:00:32.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.034+0000 7f92a4200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f929c15a990 con 0x7f929c15df80 2024-01-16T15:00:32.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.035+0000 7f92a2200640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f929c0cd470 0x7f929c15b080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48542/0 (socket says 172.21.15.143:48542) 2024-01-16T15:00:32.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.035+0000 7f92a2200640 1 -- 172.21.15.143:0/1105723282 learned_addr learned my addr 172.21.15.143:0/1105723282 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:32.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.035+0000 7f92a0e00640 1 --2- 172.21.15.143:0/1105723282 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f929c15df80 0x7f929c0ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:32.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.035+0000 7f929be00640 1 -- 172.21.15.143:0/1105723282 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2728348909 0 0) 0x7f929c15ab80 con 0x7f929c0cd470 2024-01-16T15:00:32.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.035+0000 7f929be00640 1 -- 172.21.15.143:0/1105723282 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f927c005730 con 0x7f929c0cd470 2024-01-16T15:00:32.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.035+0000 7f929be00640 1 -- 172.21.15.143:0/1105723282 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2449597276 0 0) 0x7f927c005730 con 0x7f929c0cd470 2024-01-16T15:00:32.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.036+0000 7f929be00640 1 -- 172.21.15.143:0/1105723282 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f929c15df80 msgr2=0x7f929c0ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:32.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.036+0000 7f929be00640 1 --2- 172.21.15.143:0/1105723282 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f929c15df80 0x7f929c0ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:32.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.036+0000 7f929be00640 1 -- 172.21.15.143:0/1105723282 >> v1:172.21.15.204:6789/0 conn(0x7f929c0b2250 legacy=0x7f929c0b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:32.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.036+0000 7f929be00640 1 -- 172.21.15.143:0/1105723282 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f929c0b2d40 con 0x7f929c0cd470 2024-01-16T15:00:32.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.036+0000 7f929be00640 1 -- 172.21.15.143:0/1105723282 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7f9294001670 con 0x7f929c0cd470 2024-01-16T15:00:32.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.036+0000 7f929be00640 1 -- 172.21.15.143:0/1105723282 >> v1:172.21.15.143:6789/0 conn(0x7f929c0cd470 legacy=0x7f929c15b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:32.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.036+0000 7f929be00640 1 --2- 172.21.15.143:0/1105723282 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f927c005bb0 0x7f927c005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:32.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.036+0000 7f929be00640 1 --2- 172.21.15.143:0/1105723282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f927c0069d0 0x7f927c006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:32.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.036+0000 7f929be00640 1 --2- 172.21.15.143:0/1105723282 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f929c15df80 0x7f927c007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:32.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.036+0000 7f929be00640 1 -- 172.21.15.143:0/1105723282 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927c00dc40 con 0x7f927c005bb0 2024-01-16T15:00:32.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.036+0000 7f929be00640 1 -- 172.21.15.143:0/1105723282 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927c00ddb0 con 0x7f927c0069d0 2024-01-16T15:00:32.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.036+0000 7f929be00640 1 -- 172.21.15.143:0/1105723282 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f927c00df70 con 0x7f929c15df80 2024-01-16T15:00:32.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.036+0000 7f92a1800640 1 --2- 172.21.15.143:0/1105723282 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f927c005bb0 0x7f927c005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:32.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.036+0000 7f92a0e00640 1 --2- 172.21.15.143:0/1105723282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f927c0069d0 0x7f927c006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:32.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.036+0000 7f92a1800640 1 -- 172.21.15.143:0/1105723282 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f929c15df80 msgr2=0x7f927c007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:32.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.036+0000 7f92a1800640 1 --2- 172.21.15.143:0/1105723282 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f929c15df80 0x7f927c007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:32.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.036+0000 7f92a1800640 1 -- 172.21.15.143:0/1105723282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f927c0069d0 msgr2=0x7f927c006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:32.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.036+0000 7f92a1800640 1 --2- 172.21.15.143:0/1105723282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f927c0069d0 0x7f927c006e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:32.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.036+0000 7f92a1800640 1 -- 172.21.15.143:0/1105723282 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f927c005730 con 0x7f927c005bb0 2024-01-16T15:00:32.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.037+0000 7f92a1800640 1 --2- 172.21.15.143:0/1105723282 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f927c005bb0 0x7f927c005fd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f928c001e20 tx=0x7f927c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad7ba318ea35af5e server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:32.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.038+0000 7f929be00640 1 -- 172.21.15.143:0/1105723282 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928c003b70 con 0x7f927c005bb0 2024-01-16T15:00:32.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.038+0000 7f929be00640 1 -- 172.21.15.143:0/1105723282 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f928c003cd0 con 0x7f927c005bb0 2024-01-16T15:00:32.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.038+0000 7f929be00640 1 -- 172.21.15.143:0/1105723282 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928c005310 con 0x7f927c005bb0 2024-01-16T15:00:32.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.038+0000 7f92a4200640 1 -- 172.21.15.143:0/1105723282 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f927c005bb0 msgr2=0x7f927c005fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:32.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.038+0000 7f92a4200640 1 --2- 172.21.15.143:0/1105723282 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f927c005bb0 0x7f927c005fd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f928c001e20 tx=0x7f927c007a00 comp rx=0 tx=0).stop 2024-01-16T15:00:32.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.038+0000 7f92a2200640 1 -- 172.21.15.143:0/1105723282 reap_dead start 2024-01-16T15:00:32.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.038+0000 7f92a4200640 1 -- 172.21.15.143:0/1105723282 shutdown_connections 2024-01-16T15:00:32.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.038+0000 7f92a4200640 1 -- 172.21.15.143:0/1105723282 >> 172.21.15.143:0/1105723282 conn(0x7f929c06a920 msgr2=0x7f929c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:32.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.038+0000 7f92a4200640 1 -- 172.21.15.143:0/1105723282 shutdown_connections 2024-01-16T15:00:32.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.038+0000 7f92a4200640 1 -- 172.21.15.143:0/1105723282 wait complete. 2024-01-16T15:00:32.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.040+0000 7f92a4200640 1 Processor -- start 2024-01-16T15:00:32.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.040+0000 7f92a4200640 1 -- start start 2024-01-16T15:00:32.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.041+0000 7f92a4200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f927c005bb0 0x7f929c0c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:32.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.041+0000 7f92a4200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f929c0ca6d0 0x7f929c1f34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:32.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.041+0000 7f92a4200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f929c071a00 0x7f929c071e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:32.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.041+0000 7f92a4200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929c0723c0 con 0x7f929c0ca6d0 2024-01-16T15:00:32.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.041+0000 7f92a4200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929c0c9280 con 0x7f927c005bb0 2024-01-16T15:00:32.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.041+0000 7f92a4200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f929c1f3a10 con 0x7f929c071a00 2024-01-16T15:00:32.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.041+0000 7f92a0e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f929c0ca6d0 0x7f929c1f34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:32.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.042+0000 7f92a0e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f929c0ca6d0 0x7f929c1f34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35254/0 (socket says 172.21.15.143:35254) 2024-01-16T15:00:32.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.042+0000 7f92a0e00640 1 -- 172.21.15.143:0/2621063186 learned_addr learned my addr 172.21.15.143:0/2621063186 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:32.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.042+0000 7f92a0e00640 1 -- 172.21.15.143:0/2621063186 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f929c071a00 msgr2=0x7f929c071e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:32.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.042+0000 7f92a0e00640 1 --2- 172.21.15.143:0/2621063186 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f929c071a00 0x7f929c071e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:32.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.042+0000 7f92a0e00640 1 -- 172.21.15.143:0/2621063186 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f927c005bb0 msgr2=0x7f929c0c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:32.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.042+0000 7f92a0e00640 1 --2- 172.21.15.143:0/2621063186 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f927c005bb0 0x7f929c0c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:32.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.042+0000 7f92a0e00640 1 -- 172.21.15.143:0/2621063186 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f929c0caeb0 con 0x7f929c0ca6d0 2024-01-16T15:00:32.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.042+0000 7f92a0e00640 1 --2- 172.21.15.143:0/2621063186 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f929c0ca6d0 0x7f929c1f34d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9284010860 tx=0x7f9284010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:32.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.042+0000 7f9299600640 1 -- 172.21.15.143:0/2621063186 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9284009610 con 0x7f929c0ca6d0 2024-01-16T15:00:32.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.043+0000 7f9299600640 1 -- 172.21.15.143:0/2621063186 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f928400eb60 con 0x7f929c0ca6d0 2024-01-16T15:00:32.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.043+0000 7f9299600640 1 -- 172.21.15.143:0/2621063186 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928401c400 con 0x7f929c0ca6d0 2024-01-16T15:00:32.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.043+0000 7f92a4200640 1 -- 172.21.15.143:0/2621063186 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f929c155fd0 con 0x7f929c0ca6d0 2024-01-16T15:00:32.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.043+0000 7f92a4200640 1 -- 172.21.15.143:0/2621063186 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f929c156590 con 0x7f929c0ca6d0 2024-01-16T15:00:32.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.044+0000 7f92a4200640 1 -- 172.21.15.143:0/2621063186 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f929c067470 con 0x7f929c0ca6d0 2024-01-16T15:00:32.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.045+0000 7f9299600640 1 -- 172.21.15.143:0/2621063186 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f928401c560 con 0x7f929c0ca6d0 2024-01-16T15:00:32.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.046+0000 7f9299600640 1 --2- 172.21.15.143:0/2621063186 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f927406b2c0 0x7f927406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:32.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.046+0000 7f9299600640 1 -- 172.21.15.143:0/2621063186 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f928408a1b0 con 0x7f929c0ca6d0 2024-01-16T15:00:32.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.046+0000 7f92a1800640 1 --2- 172.21.15.143:0/2621063186 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f927406b2c0 0x7f927406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:32.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.047+0000 7f92a1800640 1 --2- 172.21.15.143:0/2621063186 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f927406b2c0 0x7f927406d780 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f928c034000 tx=0x7f928c030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:32.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.051+0000 7f9299600640 1 -- 172.21.15.143:0/2621063186 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9284054fa0 con 0x7f929c0ca6d0 2024-01-16T15:00:32.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.283+0000 7f92a4200640 1 -- 172.21.15.143:0/2621063186 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f929c0cb460 con 0x7f929c0ca6d0 2024-01-16T15:00:32.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.284+0000 7f9299600640 1 -- 172.21.15.143:0/2621063186 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f92840585c0 con 0x7f929c0ca6d0 2024-01-16T15:00:32.265 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-16T15:00:32.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.286+0000 7f92a4200640 1 -- 172.21.15.143:0/2621063186 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f927406b2c0 msgr2=0x7f927406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:32.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.286+0000 7f92a4200640 1 --2- 172.21.15.143:0/2621063186 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f927406b2c0 0x7f927406d780 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f928c034000 tx=0x7f928c030000 comp rx=0 tx=0).stop 2024-01-16T15:00:32.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.286+0000 7f92a4200640 1 -- 172.21.15.143:0/2621063186 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f929c0ca6d0 msgr2=0x7f929c1f34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:32.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.286+0000 7f92a4200640 1 --2- 172.21.15.143:0/2621063186 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f929c0ca6d0 0x7f929c1f34d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9284010860 tx=0x7f9284010d20 comp rx=0 tx=0).stop 2024-01-16T15:00:32.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.287+0000 7f92a4200640 1 -- 172.21.15.143:0/2621063186 shutdown_connections 2024-01-16T15:00:32.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.287+0000 7f92a4200640 1 --2- 172.21.15.143:0/2621063186 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f927406b2c0 0x7f927406d780 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:32.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.287+0000 7f92a4200640 1 --2- 172.21.15.143:0/2621063186 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f929c071a00 0x7f929c071e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:32.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.287+0000 7f92a4200640 1 --2- 172.21.15.143:0/2621063186 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f929c0ca6d0 0x7f929c1f34d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:32.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.287+0000 7f92a4200640 1 --2- 172.21.15.143:0/2621063186 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f927c005bb0 0x7f929c0c8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:32.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.287+0000 7f92a4200640 1 -- 172.21.15.143:0/2621063186 >> 172.21.15.143:0/2621063186 conn(0x7f929c06a920 msgr2=0x7f929c15d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:32.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.287+0000 7f92a4200640 1 -- 172.21.15.143:0/2621063186 shutdown_connections 2024-01-16T15:00:32.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:32.287+0000 7f92a4200640 1 -- 172.21.15.143:0/2621063186 wait complete. 2024-01-16T15:00:32.285 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-16T15:00:32.285 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-16T15:00:32.285 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi204.front.sepia.ceph.com 2024-01-16T15:00:32.285 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-01-16T15:00:32.285 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-01-16T15:00:32.285 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-16T15:00:32.286 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-16T15:00:32.286 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-16T15:00:32.286 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-01-16T15:00:32.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:32.286 DEBUG:teuthology.orchestra.run.smithi204:> ip addr 2024-01-16T15:00:32.344 INFO:teuthology.orchestra.run.smithi204.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-16T15:00:32.344 INFO:teuthology.orchestra.run.smithi204.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-16T15:00:32.344 INFO:teuthology.orchestra.run.smithi204.stdout: inet 127.0.0.1/8 scope host lo 2024-01-16T15:00:32.345 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:00:32.345 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 ::1/128 scope host 2024-01-16T15:00:32.345 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:00:32.345 INFO:teuthology.orchestra.run.smithi204.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-16T15:00:32.345 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2a brd ff:ff:ff:ff:ff:ff 2024-01-16T15:00:32.345 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f0 2024-01-16T15:00:32.345 INFO:teuthology.orchestra.run.smithi204.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-16T15:00:32.345 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2b brd ff:ff:ff:ff:ff:ff 2024-01-16T15:00:32.345 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f1 2024-01-16T15:00:32.345 INFO:teuthology.orchestra.run.smithi204.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-16T15:00:32.345 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:aa brd ff:ff:ff:ff:ff:ff 2024-01-16T15:00:32.345 INFO:teuthology.orchestra.run.smithi204.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-16T15:00:32.345 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:ab brd ff:ff:ff:ff:ff:ff 2024-01-16T15:00:32.345 INFO:teuthology.orchestra.run.smithi204.stdout: inet 172.21.15.204/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-16T15:00:32.346 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft 42611sec preferred_lft 42611sec 2024-01-16T15:00:32.346 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::ec4:7aff:fe88:77ab/64 scope link 2024-01-16T15:00:32.346 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:00:32.346 INFO:teuthology.orchestra.run.smithi204.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-16T15:00:32.346 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 02:22:3e:e0:99:fa brd ff:ff:ff:ff:ff:ff 2024-01-16T15:00:32.346 INFO:teuthology.orchestra.run.smithi204.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-16T15:00:32.346 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:00:32.346 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::30d5:f7ff:fe0d:9796/64 scope link 2024-01-16T15:00:32.346 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:00:32.346 INFO:teuthology.orchestra.run.smithi204.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-16T15:00:32.346 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 02:22:3e:e0:99:fa brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-16T15:00:32.346 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::22:3eff:fee0:99fa/64 scope link 2024-01-16T15:00:32.346 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:00:32.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:32.347 DEBUG:teuthology.orchestra.run.smithi204:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-16T15:00:32.439 INFO:teuthology.orchestra.run.smithi204.stdout:1 2024-01-16T15:00:32.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:32.444 DEBUG:teuthology.orchestra.run.smithi204:> route 2024-01-16T15:00:32.498 INFO:teuthology.orchestra.run.smithi204.stdout:Kernel IP routing table 2024-01-16T15:00:32.498 INFO:teuthology.orchestra.run.smithi204.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-16T15:00:32.498 INFO:teuthology.orchestra.run.smithi204.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-16T15:00:32.498 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-16T15:00:32.498 INFO:teuthology.orchestra.run.smithi204.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-16T15:00:32.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:32.499 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec bash -c ' 2024-01-16T15:00:32.499 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-01-16T15:00:32.499 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-16T15:00:32.499 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-16T15:00:32.499 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-16T15:00:32.500 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-01-16T15:00:32.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:32.680 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-01-16T15:00:32.731 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-16T15:00:32.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:32.732 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list-id 2024-01-16T15:00:32.782 INFO:teuthology.orchestra.run.smithi204.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-01-16T15:00:32.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:32.784 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec bash -c ' 2024-01-16T15:00:32.784 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-01-16T15:00:32.784 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-16T15:00:32.784 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-01-16T15:00:32.784 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-01-16T15:00:32.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:32.923 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-01-16T15:00:33.013 INFO:teuthology.orchestra.run.smithi204.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-16T15:00:33.013 INFO:teuthology.orchestra.run.smithi204.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-16T15:00:33.013 INFO:teuthology.orchestra.run.smithi204.stdout: inet 127.0.0.1/8 scope host lo 2024-01-16T15:00:33.013 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:00:33.014 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 ::1/128 scope host 2024-01-16T15:00:33.014 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:00:33.014 INFO:teuthology.orchestra.run.smithi204.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-16T15:00:33.014 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 46:91:aa:d1:5c:88 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-01-16T15:00:33.014 INFO:teuthology.orchestra.run.smithi204.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-01-16T15:00:33.014 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:00:33.014 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::4491:aaff:fed1:5c88/64 scope link 2024-01-16T15:00:33.014 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:00:33.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:33.018 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-01-16T15:00:33.114 INFO:teuthology.orchestra.run.smithi204.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-16T15:00:33.114 INFO:teuthology.orchestra.run.smithi204.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-16T15:00:33.114 INFO:teuthology.orchestra.run.smithi204.stdout: inet 127.0.0.1/8 scope host lo 2024-01-16T15:00:33.114 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:00:33.114 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 ::1/128 scope host 2024-01-16T15:00:33.114 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:00:33.114 INFO:teuthology.orchestra.run.smithi204.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-16T15:00:33.114 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 46:91:aa:d1:5c:88 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-01-16T15:00:33.114 INFO:teuthology.orchestra.run.smithi204.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-01-16T15:00:33.115 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:00:33.115 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::4491:aaff:fed1:5c88/64 scope link 2024-01-16T15:00:33.115 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:00:33.119 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-01-16T15:00:33.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:33.119 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec bash -c ' 2024-01-16T15:00:33.119 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-01-16T15:00:33.119 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-01-16T15:00:33.119 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-01-16T15:00:33.119 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-01-16T15:00:33.119 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-01-16T15:00:33.119 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-01-16T15:00:33.119 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-01-16T15:00:33.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:33.384 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec bash -c ' 2024-01-16T15:00:33.384 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-01-16T15:00:33.384 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set brx.1 up 2024-01-16T15:00:33.384 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set dev brx.1 master ceph-brx 2024-01-16T15:00:33.384 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-01-16T15:00:33.554 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-16T15:00:33.555 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-01-16T15:00:33.600 INFO:teuthology.orchestra.run.smithi204.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-01-16T15:00:33.600 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-16T15:00:33.600 DEBUG:teuthology.orchestra.run.smithi204:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-01-16T15:00:33.693 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-16T15:00:33.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:33.693 DEBUG:teuthology.orchestra.run.smithi204:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-01-16T15:00:33.833 INFO:teuthology.orchestra.run.smithi204.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-01-16T15:00:33.833 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-01-16T15:00:33.833 INFO:teuthology.orchestra.run.smithi204.stdout:invalid new device string format 2024-01-16T15:00:33.855 INFO:teuthology.orchestra.run.smithi204.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-01-16T15:00:33.855 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-01-16T15:00:33.855 INFO:teuthology.orchestra.run.smithi204.stdout:invalid new device string format 2024-01-16T15:00:33.855 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: resolved to: "172.21.15.143:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-01-16T15:00:33.856 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-16T15:00:33.882 INFO:teuthology.orchestra.run.smithi204.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-01-16T15:00:33.882 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-01-16T15:00:33.883 INFO:teuthology.orchestra.run.smithi204.stdout:invalid new device string format 2024-01-16T15:00:33.883 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: resolved to: "172.21.15.143:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-01-16T15:00:33.883 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-16T15:00:33.883 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: resolved to: "172.21.15.143:6789,172.21.15.204:6789,172.21.15.204:6790" 2024-01-16T15:00:33.883 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.143:6789,172.21.15.204:6789,172.21.15.204:6790:/ 2024-01-16T15:00:33.883 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0" will pass to kernel 2024-01-16T15:00:33.885 INFO:teuthology.orchestra.run.smithi204.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-01-16T15:00:33.885 INFO:teuthology.orchestra.run.smithi204.stdout: You just mounted a file system that supports labels which does not 2024-01-16T15:00:33.885 INFO:teuthology.orchestra.run.smithi204.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-01-16T15:00:33.885 INFO:teuthology.orchestra.run.smithi204.stdout: applications will generate AVC messages and not be allowed access to 2024-01-16T15:00:33.885 INFO:teuthology.orchestra.run.smithi204.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-01-16T15:00:33.891 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-16T15:00:33.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:33.891 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-01-16T15:00:33.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:33.975 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-16T15:00:34.032 INFO:teuthology.orchestra.run.smithi204.stdout:client4445 2024-01-16T15:00:34.032 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-16T15:00:34.032 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T15:00:34.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:34.082 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-16T15:00:34.138 INFO:teuthology.orchestra.run.smithi204.stdout:client4445 2024-01-16T15:00:34.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:34.138 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-16T15:00:34.191 INFO:teuthology.orchestra.run.smithi204.stdout:client4445 2024-01-16T15:00:34.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:34.191 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0.client4445/status') 2024-01-16T15:00:34.275 INFO:teuthology.orchestra.run.smithi204.stdout:instance: client.4445 (3)192.168.0.2:0/3854923269 2024-01-16T15:00:34.275 INFO:teuthology.orchestra.run.smithi204.stdout:blocklisted: false 2024-01-16T15:00:34.276 INFO:teuthology.orchestra.run.smithi204.stderr:0+1 records in 2024-01-16T15:00:34.276 INFO:teuthology.orchestra.run.smithi204.stderr:0+1 records out 2024-01-16T15:00:34.276 INFO:teuthology.orchestra.run.smithi204.stderr:69 bytes copied, 6.3559e-05 s, 1.1 MB/s 2024-01-16T15:00:34.280 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-01-16T15:00:34.295 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T15:00:34.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.529+0000 7fdd92200640 1 Processor -- start 2024-01-16T15:00:34.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.529+0000 7fdd92200640 1 -- start start 2024-01-16T15:00:34.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.531+0000 7fdd92200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdd8c15c7f0 0x7fdd8c06db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:34.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.531+0000 7fdd92200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd8c0ce6c0 0x7fdd8c0cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:34.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.531+0000 7fdd92200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd8c0cece0 con 0x7fdd8c0cf450 2024-01-16T15:00:34.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.531+0000 7fdd92200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd8c0cfe40 con 0x7fdd8c15c7f0 2024-01-16T15:00:34.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.531+0000 7fdd92200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8c156010 con 0x7fdd8c0ce6c0 2024-01-16T15:00:34.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.531+0000 7fdd8aa00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fdd8c0cf450 0x7fdd8c0cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48556/0 (socket says 172.21.15.143:48556) 2024-01-16T15:00:34.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.531+0000 7fdd8aa00640 1 -- 172.21.15.143:0/1890629118 learned_addr learned my addr 172.21.15.143:0/1890629118 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:34.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.532+0000 7fdd8b400640 1 --2- 172.21.15.143:0/1890629118 >> v2:172.21.15.204:3300/0 conn(0x7fdd8c15c7f0 0x7fdd8c06db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:34.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.532+0000 7fdd8be00640 1 --2- 172.21.15.143:0/1890629118 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd8c0ce6c0 0x7fdd8c0cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:34.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.532+0000 7fdd8a000640 1 -- 172.21.15.143:0/1890629118 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2633984985 0 0) 0x7fdd8c0cece0 con 0x7fdd8c0cf450 2024-01-16T15:00:34.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.532+0000 7fdd8a000640 1 -- 172.21.15.143:0/1890629118 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd6c003520 con 0x7fdd8c0cf450 2024-01-16T15:00:34.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.532+0000 7fdd8a000640 1 -- 172.21.15.143:0/1890629118 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2840411835 0 0) 0x7fdd6c003520 con 0x7fdd8c0cf450 2024-01-16T15:00:34.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.532+0000 7fdd8a000640 1 -- 172.21.15.143:0/1890629118 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd8c0ce6c0 msgr2=0x7fdd8c0cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:34.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.532+0000 7fdd8a000640 1 --2- 172.21.15.143:0/1890629118 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd8c0ce6c0 0x7fdd8c0cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:34.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.532+0000 7fdd8a000640 1 -- 172.21.15.143:0/1890629118 >> v2:172.21.15.204:3300/0 conn(0x7fdd8c15c7f0 msgr2=0x7fdd8c06db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:34.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.532+0000 7fdd8a000640 1 --2- 172.21.15.143:0/1890629118 >> v2:172.21.15.204:3300/0 conn(0x7fdd8c15c7f0 0x7fdd8c06db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:34.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.532+0000 7fdd8a000640 1 -- 172.21.15.143:0/1890629118 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd8c06d040 con 0x7fdd8c0cf450 2024-01-16T15:00:34.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.532+0000 7fdd8a000640 1 -- 172.21.15.143:0/1890629118 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7fdd74001670 con 0x7fdd8c0cf450 2024-01-16T15:00:34.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.532+0000 7fdd8be00640 1 --2- 172.21.15.143:0/1890629118 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd8c0ce6c0 0x7fdd8c0cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T15:00:34.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.532+0000 7fdd8a000640 1 -- 172.21.15.143:0/1890629118 >> v1:172.21.15.143:6789/0 conn(0x7fdd8c0cf450 legacy=0x7fdd8c0cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:34.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.532+0000 7fdd8a000640 1 --2- 172.21.15.143:0/1890629118 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd8c0ce6c0 0x7fdd6c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:34.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.532+0000 7fdd8a000640 1 --2- 172.21.15.143:0/1890629118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd6c003f00 0x7fdd6c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:34.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.532+0000 7fdd8a000640 1 --2- 172.21.15.143:0/1890629118 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd6c004d00 0x7fdd6c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:34.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.532+0000 7fdd8a000640 1 -- 172.21.15.143:0/1890629118 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8c0cfe40 con 0x7fdd6c004d00 2024-01-16T15:00:34.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.532+0000 7fdd8a000640 1 -- 172.21.15.143:0/1890629118 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8c156010 con 0x7fdd6c003f00 2024-01-16T15:00:34.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.532+0000 7fdd8a000640 1 -- 172.21.15.143:0/1890629118 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c00ba00 con 0x7fdd8c0ce6c0 2024-01-16T15:00:34.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.533+0000 7fdd8be00640 1 --2- 172.21.15.143:0/1890629118 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd6c004d00 0x7fdd6c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:34.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.533+0000 7fdd8b400640 1 --2- 172.21.15.143:0/1890629118 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd8c0ce6c0 0x7fdd6c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:34.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.533+0000 7fdd8aa00640 1 --2- 172.21.15.143:0/1890629118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd6c003f00 0x7fdd6c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:34.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.533+0000 7fdd8be00640 1 -- 172.21.15.143:0/1890629118 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd8c0ce6c0 msgr2=0x7fdd6c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:34.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.533+0000 7fdd8be00640 1 --2- 172.21.15.143:0/1890629118 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd8c0ce6c0 0x7fdd6c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:34.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.533+0000 7fdd8be00640 1 -- 172.21.15.143:0/1890629118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd6c003f00 msgr2=0x7fdd6c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:34.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.533+0000 7fdd8be00640 1 --2- 172.21.15.143:0/1890629118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd6c003f00 0x7fdd6c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:34.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.533+0000 7fdd8be00640 1 -- 172.21.15.143:0/1890629118 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6c003520 con 0x7fdd6c004d00 2024-01-16T15:00:34.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.534+0000 7fdd8be00640 1 --2- 172.21.15.143:0/1890629118 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd6c004d00 0x7fdd6c005180 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdd840098a0 tx=0x7fdd6c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f3662ac7a3885af server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:34.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.534+0000 7fdd8a000640 1 -- 172.21.15.143:0/1890629118 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd84004230 con 0x7fdd6c004d00 2024-01-16T15:00:34.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.534+0000 7fdd8a000640 1 -- 172.21.15.143:0/1890629118 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd84004390 con 0x7fdd6c004d00 2024-01-16T15:00:34.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.534+0000 7fdd8a000640 1 -- 172.21.15.143:0/1890629118 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd84003830 con 0x7fdd6c004d00 2024-01-16T15:00:34.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.535+0000 7fdd92200640 1 -- 172.21.15.143:0/1890629118 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd6c004d00 msgr2=0x7fdd6c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:34.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.535+0000 7fdd92200640 1 --2- 172.21.15.143:0/1890629118 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd6c004d00 0x7fdd6c005180 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdd840098a0 tx=0x7fdd6c0057f0 comp rx=0 tx=0).stop 2024-01-16T15:00:34.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.535+0000 7fdd8be00640 1 -- 172.21.15.143:0/1890629118 reap_dead start 2024-01-16T15:00:34.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.535+0000 7fdd92200640 1 -- 172.21.15.143:0/1890629118 shutdown_connections 2024-01-16T15:00:34.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.535+0000 7fdd92200640 1 -- 172.21.15.143:0/1890629118 >> 172.21.15.143:0/1890629118 conn(0x7fdd8c06a920 msgr2=0x7fdd8c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:34.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.535+0000 7fdd92200640 1 -- 172.21.15.143:0/1890629118 shutdown_connections 2024-01-16T15:00:34.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.535+0000 7fdd92200640 1 -- 172.21.15.143:0/1890629118 wait complete. 2024-01-16T15:00:34.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.537+0000 7fdd92200640 1 Processor -- start 2024-01-16T15:00:34.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.537+0000 7fdd92200640 1 -- start start 2024-01-16T15:00:34.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.538+0000 7fdd92200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd6c004d00 0x7fdd8c1f4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:34.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.538+0000 7fdd92200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd8c14f4d0 0x7fdd8c156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:34.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.538+0000 7fdd92200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd8c14f930 0x7fdd8c154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:34.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.538+0000 7fdd92200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8c1f4940 con 0x7fdd8c14f930 2024-01-16T15:00:34.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.538+0000 7fdd92200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8c1f52a0 con 0x7fdd8c14f4d0 2024-01-16T15:00:34.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.538+0000 7fdd92200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8c1f4310 con 0x7fdd6c004d00 2024-01-16T15:00:34.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.538+0000 7fdd8be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd8c14f930 0x7fdd8c154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:34.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.538+0000 7fdd8be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd8c14f930 0x7fdd8c154370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35278/0 (socket says 172.21.15.143:35278) 2024-01-16T15:00:34.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.538+0000 7fdd8be00640 1 -- 172.21.15.143:0/3271952605 learned_addr learned my addr 172.21.15.143:0/3271952605 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:34.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.539+0000 7fdd8b400640 1 --2- 172.21.15.143:0/3271952605 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd6c004d00 0x7fdd8c1f4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:34.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.539+0000 7fdd8aa00640 1 --2- 172.21.15.143:0/3271952605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd8c14f4d0 0x7fdd8c156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:34.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.539+0000 7fdd8be00640 1 -- 172.21.15.143:0/3271952605 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd6c004d00 msgr2=0x7fdd8c1f4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:34.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.539+0000 7fdd8be00640 1 --2- 172.21.15.143:0/3271952605 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd6c004d00 0x7fdd8c1f4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:34.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.539+0000 7fdd8be00640 1 -- 172.21.15.143:0/3271952605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd8c14f4d0 msgr2=0x7fdd8c156150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:34.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.539+0000 7fdd8be00640 1 --2- 172.21.15.143:0/3271952605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd8c14f4d0 0x7fdd8c156150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:34.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.539+0000 7fdd8be00640 1 -- 172.21.15.143:0/3271952605 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd8c156ba0 con 0x7fdd8c14f930 2024-01-16T15:00:34.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.539+0000 7fdd8be00640 1 --2- 172.21.15.143:0/3271952605 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd8c14f930 0x7fdd8c154370 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdd840099c0 tx=0x7fdd84004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:34.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.539+0000 7fdd83400640 1 -- 172.21.15.143:0/3271952605 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd84030630 con 0x7fdd8c14f930 2024-01-16T15:00:34.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.540+0000 7fdd83400640 1 -- 172.21.15.143:0/3271952605 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd84030790 con 0x7fdd8c14f930 2024-01-16T15:00:34.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.540+0000 7fdd83400640 1 -- 172.21.15.143:0/3271952605 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd84030b00 con 0x7fdd8c14f930 2024-01-16T15:00:34.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.540+0000 7fdd92200640 1 -- 172.21.15.143:0/3271952605 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd8c156e80 con 0x7fdd8c14f930 2024-01-16T15:00:34.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.540+0000 7fdd92200640 1 -- 172.21.15.143:0/3271952605 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd8c15a8e0 con 0x7fdd8c14f930 2024-01-16T15:00:34.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.541+0000 7fdd92200640 1 -- 172.21.15.143:0/3271952605 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd8c067470 con 0x7fdd8c14f930 2024-01-16T15:00:34.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.542+0000 7fdd83400640 1 -- 172.21.15.143:0/3271952605 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fdd84030c60 con 0x7fdd8c14f930 2024-01-16T15:00:34.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.542+0000 7fdd83400640 1 --2- 172.21.15.143:0/3271952605 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fdd6406b2c0 0x7fdd6406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:34.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.543+0000 7fdd83400640 1 -- 172.21.15.143:0/3271952605 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdd84000e40 con 0x7fdd8c14f930 2024-01-16T15:00:34.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.543+0000 7fdd8b400640 1 --2- 172.21.15.143:0/3271952605 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fdd6406b2c0 0x7fdd6406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:34.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.543+0000 7fdd8b400640 1 --2- 172.21.15.143:0/3271952605 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fdd6406b2c0 0x7fdd6406d780 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c007520 tx=0x7fdd7c00b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:34.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.547+0000 7fdd83400640 1 -- 172.21.15.143:0/3271952605 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdd84075cc0 con 0x7fdd8c14f930 2024-01-16T15:00:34.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.745+0000 7fdd92200640 1 -- 172.21.15.143:0/3271952605 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd8c0cd730 con 0x7fdd8c14f930 2024-01-16T15:00:34.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.748+0000 7fdd83400640 1 -- 172.21.15.143:0/3271952605 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16565 (secure 0 0 0) 0x7fdd84006e10 con 0x7fdd8c14f930 2024-01-16T15:00:34.729 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:00:34.729 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0","created":"2024-01-16T14:59:38.532601+0000","modified":"2024-01-16T15:00:04.543620+0000","last_up_change":"2024-01-16T14:59:50.983699+0000","last_in_change":"2024-01-16T14:59:42.056297+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:59:51.727720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T15:00:01.010472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T15:00:02.041228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f4c8891d-ab29-472a-907d-8446f20f7012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6801","nonce":1569293626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6805","nonce":1569293626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6814","nonce":1569293626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6809","nonce":1569293626}]},"public_addr":"172.21.15.143:6801/1569293626","cluster_addr":"172.21.15.143:6805/1569293626","heartbeat_back_addr":"172.21.15.143:6814/1569293626","heartbeat_front_addr":"172.21.15.143:6809/1569293626","state":["exists","up"]},{"osd":1,"uuid":"c4a2788d-8b89-4992-82b2-7f4b9debd509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6804","nonce":1276841168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6808","nonce":1276841168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6813","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6815","nonce":1276841168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6811","nonce":1276841168}]},"public_addr":"172.21.15.143:6804/1276841168","cluster_addr":"172.21.15.143:6808/1276841168","heartbeat_back_addr":"172.21.15.143:6815/1276841168","heartbeat_front_addr":"172.21.15.143:6811/1276841168","state":["exists","up"]},{"osd":2,"uuid":"64645c1d-31b2-4794-b0f2-7ccf8df30393","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6817","nonce":1217670170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6819","nonce":1217670170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6823","nonce":1217670170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6821","nonce":1217670170}]},"public_addr":"172.21.15.143:6817/1217670170","cluster_addr":"172.21.15.143:6819/1217670170","heartbeat_back_addr":"172.21.15.143:6823/1217670170","heartbeat_front_addr":"172.21.15.143:6821/1217670170","state":["exists","up"]},{"osd":3,"uuid":"73674129-a84e-4d31-8247-7d0179ae6df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6825","nonce":3414188012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6827","nonce":3414188012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6831","nonce":3414188012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6829","nonce":3414188012}]},"public_addr":"172.21.15.143:6825/3414188012","cluster_addr":"172.21.15.143:6827/3414188012","heartbeat_back_addr":"172.21.15.143:6831/3414188012","heartbeat_front_addr":"172.21.15.143:6829/3414188012","state":["exists","up"]},{"osd":4,"uuid":"cea28c9b-2f1c-4087-b415-9960789131d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6811","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6812","nonce":1070255716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6816","nonce":1070255716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6823","nonce":1070255716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6820","nonce":1070255716}]},"public_addr":"172.21.15.204:6812/1070255716","cluster_addr":"172.21.15.204:6816/1070255716","heartbeat_back_addr":"172.21.15.204:6823/1070255716","heartbeat_front_addr":"172.21.15.204:6820/1070255716","state":["exists","up"]},{"osd":5,"uuid":"33b63560-5574-4e78-983f-b398d67a2cb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6809","nonce":3605417837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6813","nonce":3605417837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6819","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6821","nonce":3605417837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6817","nonce":3605417837}]},"public_addr":"172.21.15.204:6809/3605417837","cluster_addr":"172.21.15.204:6813/3605417837","heartbeat_back_addr":"172.21.15.204:6821/3605417837","heartbeat_front_addr":"172.21.15.204:6817/3605417837","state":["exists","up"]},{"osd":6,"uuid":"df397427-8940-4eee-9c79-eece669d735d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6801","nonce":1085000534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6803","nonce":1085000534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6808","nonce":1085000534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6805","nonce":1085000534}]},"public_addr":"172.21.15.204:6801/1085000534","cluster_addr":"172.21.15.204:6803/1085000534","heartbeat_back_addr":"172.21.15.204:6808/1085000534","heartbeat_front_addr":"172.21.15.204:6805/1085000534","state":["exists","up"]},{"osd":7,"uuid":"a2e5a15c-8312-405a-ac02-f7e1fc9df2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6825","nonce":1337220609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6827","nonce":1337220609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6831","nonce":1337220609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6829","nonce":1337220609}]},"public_addr":"172.21.15.204:6825/1337220609","cluster_addr":"172.21.15.204:6827/1337220609","heartbeat_back_addr":"172.21.15.204:6831/1337220609","heartbeat_front_addr":"172.21.15.204:6829/1337220609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:49.984507+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.395306+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.632998+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.373627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.342920+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.438366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.331926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.406657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T15:00:34.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.750+0000 7fdd92200640 1 -- 172.21.15.143:0/3271952605 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fdd6406b2c0 msgr2=0x7fdd6406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:34.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.750+0000 7fdd92200640 1 --2- 172.21.15.143:0/3271952605 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fdd6406b2c0 0x7fdd6406d780 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c007520 tx=0x7fdd7c00b040 comp rx=0 tx=0).stop 2024-01-16T15:00:34.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.750+0000 7fdd92200640 1 -- 172.21.15.143:0/3271952605 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd8c14f930 msgr2=0x7fdd8c154370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:34.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.750+0000 7fdd92200640 1 --2- 172.21.15.143:0/3271952605 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd8c14f930 0x7fdd8c154370 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdd840099c0 tx=0x7fdd84004730 comp rx=0 tx=0).stop 2024-01-16T15:00:34.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.750+0000 7fdd92200640 1 -- 172.21.15.143:0/3271952605 shutdown_connections 2024-01-16T15:00:34.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.750+0000 7fdd92200640 1 --2- 172.21.15.143:0/3271952605 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fdd6406b2c0 0x7fdd6406d780 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:34.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.750+0000 7fdd92200640 1 --2- 172.21.15.143:0/3271952605 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd8c14f930 0x7fdd8c154370 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:34.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.750+0000 7fdd92200640 1 --2- 172.21.15.143:0/3271952605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd8c14f4d0 0x7fdd8c156150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:34.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.750+0000 7fdd92200640 1 --2- 172.21.15.143:0/3271952605 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd6c004d00 0x7fdd8c1f4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:34.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.750+0000 7fdd92200640 1 -- 172.21.15.143:0/3271952605 >> 172.21.15.143:0/3271952605 conn(0x7fdd8c06a920 msgr2=0x7fdd8c0cdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:34.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.751+0000 7fdd92200640 1 -- 172.21.15.143:0/3271952605 shutdown_connections 2024-01-16T15:00:34.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.751+0000 7fdd92200640 1 -- 172.21.15.143:0/3271952605 wait complete. 2024-01-16T15:00:34.749 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-16T14:59:51.727720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-16T15:00:01.010472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-16T15:00:02.041228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-16T15:00:34.750 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-16T15:00:34.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.982+0000 7fd1d4a00640 1 Processor -- start 2024-01-16T15:00:34.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.982+0000 7fd1d4a00640 1 -- start start 2024-01-16T15:00:34.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.982+0000 7fd1d4a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1cc072160 0x7fd1cc15e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:34.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.982+0000 7fd1d4a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1cc15ec90 con 0x7fd1cc0c49f0 2024-01-16T15:00:34.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.983+0000 7fd1d4a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1cc0cd530 con 0x7fd1cc06de10 2024-01-16T15:00:34.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.983+0000 7fd1d4a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc0c4ed0 con 0x7fd1cc072160 2024-01-16T15:00:34.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.983+0000 7fd1d1600640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd1cc0c49f0 0x7fd1cc06d700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49178/0 (socket says 172.21.15.143:49178) 2024-01-16T15:00:34.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.983+0000 7fd1d1600640 1 -- 172.21.15.143:0/2019278410 learned_addr learned my addr 172.21.15.143:0/2019278410 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:34.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.983+0000 7fd1d2000640 1 --2- 172.21.15.143:0/2019278410 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1cc072160 0x7fd1cc15e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:34.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.983+0000 7fd1d0c00640 1 -- 172.21.15.143:0/2019278410 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 532822343 0 0) 0x7fd1b4001110 con 0x7fd1cc0c49f0 2024-01-16T15:00:34.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.983+0000 7fd1d0c00640 1 -- 172.21.15.143:0/2019278410 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1ac003520 con 0x7fd1cc0c49f0 2024-01-16T15:00:34.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.984+0000 7fd1d0c00640 1 -- 172.21.15.143:0/2019278410 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2836318940 0 0) 0x7fd1ac003520 con 0x7fd1cc0c49f0 2024-01-16T15:00:34.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.984+0000 7fd1d0c00640 1 -- 172.21.15.143:0/2019278410 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1cc072160 msgr2=0x7fd1cc15e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:34.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.984+0000 7fd1d0c00640 1 --2- 172.21.15.143:0/2019278410 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1cc072160 0x7fd1cc15e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:34.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.984+0000 7fd1d0c00640 1 -- 172.21.15.143:0/2019278410 >> v1:172.21.15.204:6789/0 conn(0x7fd1cc06de10 legacy=0x7fd1cc0705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:34.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.984+0000 7fd1d0c00640 1 -- 172.21.15.143:0/2019278410 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1cc0c51b0 con 0x7fd1cc0c49f0 2024-01-16T15:00:34.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.984+0000 7fd1d0c00640 1 -- 172.21.15.143:0/2019278410 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7fd1b4001800 con 0x7fd1cc0c49f0 2024-01-16T15:00:34.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.984+0000 7fd1d0c00640 1 -- 172.21.15.143:0/2019278410 >> v1:172.21.15.143:6789/0 conn(0x7fd1cc0c49f0 legacy=0x7fd1cc06d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:34.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.984+0000 7fd1d0c00640 1 --2- 172.21.15.143:0/2019278410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1ac0039a0 0x7fd1ac003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:34.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.984+0000 7fd1d0c00640 1 --2- 172.21.15.143:0/2019278410 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1ac0047c0 0x7fd1ac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:34.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.984+0000 7fd1d0c00640 1 --2- 172.21.15.143:0/2019278410 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1cc072160 0x7fd1ac005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:34.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.984+0000 7fd1d0c00640 1 -- 172.21.15.143:0/2019278410 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc0c4ed0 con 0x7fd1ac0047c0 2024-01-16T15:00:34.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.984+0000 7fd1d0c00640 1 -- 172.21.15.143:0/2019278410 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac00ba60 con 0x7fd1ac0039a0 2024-01-16T15:00:34.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.984+0000 7fd1d0c00640 1 -- 172.21.15.143:0/2019278410 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac00de30 con 0x7fd1cc072160 2024-01-16T15:00:34.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.984+0000 7fd1d1600640 1 --2- 172.21.15.143:0/2019278410 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1ac0047c0 0x7fd1ac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:34.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.985+0000 7fd1d2a00640 1 --2- 172.21.15.143:0/2019278410 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1cc072160 0x7fd1ac005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:34.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.985+0000 7fd1d1600640 1 -- 172.21.15.143:0/2019278410 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1cc072160 msgr2=0x7fd1ac005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:34.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.985+0000 7fd1d1600640 1 --2- 172.21.15.143:0/2019278410 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1cc072160 0x7fd1ac005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:34.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.985+0000 7fd1d1600640 1 -- 172.21.15.143:0/2019278410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1ac0039a0 msgr2=0x7fd1ac003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:34.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.985+0000 7fd1d1600640 1 --2- 172.21.15.143:0/2019278410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1ac0039a0 0x7fd1ac003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:34.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.985+0000 7fd1d1600640 1 -- 172.21.15.143:0/2019278410 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ac003520 con 0x7fd1ac0047c0 2024-01-16T15:00:34.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.986+0000 7fd1d1600640 1 --2- 172.21.15.143:0/2019278410 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1ac0047c0 0x7fd1ac004c40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4007fa0 tx=0x7fd1ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77813623aa0fd53c server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:34.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.986+0000 7fd1d0c00640 1 -- 172.21.15.143:0/2019278410 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4005ad0 con 0x7fd1ac0047c0 2024-01-16T15:00:34.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.986+0000 7fd1d0c00640 1 -- 172.21.15.143:0/2019278410 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b4005c30 con 0x7fd1ac0047c0 2024-01-16T15:00:34.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.986+0000 7fd1d0c00640 1 -- 172.21.15.143:0/2019278410 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b402c770 con 0x7fd1ac0047c0 2024-01-16T15:00:34.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.987+0000 7fd1d4a00640 1 -- 172.21.15.143:0/2019278410 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1ac0047c0 msgr2=0x7fd1ac004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:34.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.987+0000 7fd1d4a00640 1 --2- 172.21.15.143:0/2019278410 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1ac0047c0 0x7fd1ac004c40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4007fa0 tx=0x7fd1ac0057f0 comp rx=0 tx=0).stop 2024-01-16T15:00:34.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.987+0000 7fd1d2a00640 1 -- 172.21.15.143:0/2019278410 reap_dead start 2024-01-16T15:00:34.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.987+0000 7fd1d4a00640 1 -- 172.21.15.143:0/2019278410 shutdown_connections 2024-01-16T15:00:34.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.987+0000 7fd1d4a00640 1 -- 172.21.15.143:0/2019278410 >> 172.21.15.143:0/2019278410 conn(0x7fd1cc0b0c50 msgr2=0x7fd1cc0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:34.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.987+0000 7fd1d4a00640 1 -- 172.21.15.143:0/2019278410 shutdown_connections 2024-01-16T15:00:34.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.987+0000 7fd1d4a00640 1 -- 172.21.15.143:0/2019278410 wait complete. 2024-01-16T15:00:34.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.989+0000 7fd1d4a00640 1 Processor -- start 2024-01-16T15:00:34.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.989+0000 7fd1d4a00640 1 -- start start 2024-01-16T15:00:34.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.990+0000 7fd1d4a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1ac0047c0 0x7fd1cc14be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:34.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.990+0000 7fd1d4a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1cc15a380 0x7fd1cc15a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:34.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.990+0000 7fd1d4a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1cc0c86f0 0x7fd1cc0c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:34.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.990+0000 7fd1d4a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc0c90b0 con 0x7fd1cc0c86f0 2024-01-16T15:00:34.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.990+0000 7fd1d4a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc15ae40 con 0x7fd1cc15a380 2024-01-16T15:00:34.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.990+0000 7fd1d4a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc15b120 con 0x7fd1ac0047c0 2024-01-16T15:00:34.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.990+0000 7fd1d2a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1cc0c86f0 0x7fd1cc0c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:34.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.990+0000 7fd1d2a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1cc0c86f0 0x7fd1cc0c8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43112/0 (socket says 172.21.15.143:43112) 2024-01-16T15:00:34.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.990+0000 7fd1d2a00640 1 -- 172.21.15.143:0/2315661173 learned_addr learned my addr 172.21.15.143:0/2315661173 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:34.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.990+0000 7fd1d2000640 1 --2- 172.21.15.143:0/2315661173 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1ac0047c0 0x7fd1cc14be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:34.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.990+0000 7fd1d1600640 1 --2- 172.21.15.143:0/2315661173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1cc15a380 0x7fd1cc15a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:34.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.990+0000 7fd1d2a00640 1 -- 172.21.15.143:0/2315661173 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1ac0047c0 msgr2=0x7fd1cc14be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:34.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.990+0000 7fd1d2a00640 1 --2- 172.21.15.143:0/2315661173 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1ac0047c0 0x7fd1cc14be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:34.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.990+0000 7fd1d2a00640 1 -- 172.21.15.143:0/2315661173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1cc15a380 msgr2=0x7fd1cc15a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:34.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.990+0000 7fd1d2a00640 1 --2- 172.21.15.143:0/2315661173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1cc15a380 0x7fd1cc15a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:34.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.990+0000 7fd1d2a00640 1 -- 172.21.15.143:0/2315661173 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1cc15b400 con 0x7fd1cc0c86f0 2024-01-16T15:00:34.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.991+0000 7fd1d2a00640 1 --2- 172.21.15.143:0/2315661173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1cc0c86f0 0x7fd1cc0c8b70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc0b51a0 tx=0x7fd1c400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:34.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.991+0000 7fd1ca000640 1 -- 172.21.15.143:0/2315661173 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4003970 con 0x7fd1cc0c86f0 2024-01-16T15:00:34.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.991+0000 7fd1ca000640 1 -- 172.21.15.143:0/2315661173 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c4003ad0 con 0x7fd1cc0c86f0 2024-01-16T15:00:34.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.991+0000 7fd1ca000640 1 -- 172.21.15.143:0/2315661173 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4003e80 con 0x7fd1cc0c86f0 2024-01-16T15:00:34.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.991+0000 7fd1d4a00640 1 -- 172.21.15.143:0/2315661173 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1cc15b6e0 con 0x7fd1cc0c86f0 2024-01-16T15:00:34.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.991+0000 7fd1d4a00640 1 -- 172.21.15.143:0/2315661173 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1cc14c360 con 0x7fd1cc0c86f0 2024-01-16T15:00:34.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.992+0000 7fd1d4a00640 1 -- 172.21.15.143:0/2315661173 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1cc04e8b0 con 0x7fd1cc0c86f0 2024-01-16T15:00:34.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.994+0000 7fd1ca000640 1 -- 172.21.15.143:0/2315661173 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fd1c40050d0 con 0x7fd1cc0c86f0 2024-01-16T15:00:34.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.995+0000 7fd1ca000640 1 --2- 172.21.15.143:0/2315661173 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd1a406b2c0 0x7fd1a406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:34.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.995+0000 7fd1ca000640 1 -- 172.21.15.143:0/2315661173 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd1c4089a50 con 0x7fd1cc0c86f0 2024-01-16T15:00:34.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.995+0000 7fd1d2000640 1 --2- 172.21.15.143:0/2315661173 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd1a406b2c0 0x7fd1a406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:34.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.996+0000 7fd1d2000640 1 --2- 172.21.15.143:0/2315661173 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd1a406b2c0 0x7fd1a406d780 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc004770 tx=0x7fd1bc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:34.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:34.999+0000 7fd1ca000640 1 -- 172.21.15.143:0/2315661173 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd1c4054820 con 0x7fd1cc0c86f0 2024-01-16T15:00:35.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.189+0000 7fd1d4a00640 1 -- 172.21.15.143:0/2315661173 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd1cc15dab0 con 0x7fd1cc0c86f0 2024-01-16T15:00:35.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.190+0000 7fd1ca000640 1 -- 172.21.15.143:0/2315661173 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd1c4057e40 con 0x7fd1cc0c86f0 2024-01-16T15:00:35.171 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-16T15:00:35.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.192+0000 7fd1d4a00640 1 -- 172.21.15.143:0/2315661173 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd1a406b2c0 msgr2=0x7fd1a406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:35.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.192+0000 7fd1d4a00640 1 --2- 172.21.15.143:0/2315661173 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd1a406b2c0 0x7fd1a406d780 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc004770 tx=0x7fd1bc008040 comp rx=0 tx=0).stop 2024-01-16T15:00:35.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.192+0000 7fd1d4a00640 1 -- 172.21.15.143:0/2315661173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1cc0c86f0 msgr2=0x7fd1cc0c8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:35.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.192+0000 7fd1d4a00640 1 --2- 172.21.15.143:0/2315661173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1cc0c86f0 0x7fd1cc0c8b70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc0b51a0 tx=0x7fd1c400ec10 comp rx=0 tx=0).stop 2024-01-16T15:00:35.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.192+0000 7fd1d4a00640 1 -- 172.21.15.143:0/2315661173 shutdown_connections 2024-01-16T15:00:35.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.192+0000 7fd1d4a00640 1 --2- 172.21.15.143:0/2315661173 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd1a406b2c0 0x7fd1a406d780 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:35.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.192+0000 7fd1d4a00640 1 --2- 172.21.15.143:0/2315661173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1cc0c86f0 0x7fd1cc0c8b70 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:35.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.192+0000 7fd1d4a00640 1 --2- 172.21.15.143:0/2315661173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1cc15a380 0x7fd1cc15a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:35.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.192+0000 7fd1d4a00640 1 --2- 172.21.15.143:0/2315661173 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1ac0047c0 0x7fd1cc14be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:35.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.192+0000 7fd1d4a00640 1 -- 172.21.15.143:0/2315661173 >> 172.21.15.143:0/2315661173 conn(0x7fd1cc0b0c50 msgr2=0x7fd1cc06c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:35.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.193+0000 7fd1d4a00640 1 -- 172.21.15.143:0/2315661173 shutdown_connections 2024-01-16T15:00:35.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.193+0000 7fd1d4a00640 1 -- 172.21.15.143:0/2315661173 wait complete. 2024-01-16T15:00:35.193 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-16T15:00:35.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.424+0000 7fa511e00640 1 Processor -- start 2024-01-16T15:00:35.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.425+0000 7fa511e00640 1 -- start start 2024-01-16T15:00:35.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.427+0000 7fa511e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa50c06e950 0x7fa50c161bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:35.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.427+0000 7fa511e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa50c0ac840 con 0x7fa50c06b780 2024-01-16T15:00:35.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.427+0000 7fa511e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa50c0ad6a0 con 0x7fa50c06bb60 2024-01-16T15:00:35.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.427+0000 7fa511e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa50c162100 con 0x7fa50c06e950 2024-01-16T15:00:35.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.427+0000 7fa50aa00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fa50c06b780 0x7fa50c0b2f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49184/0 (socket says 172.21.15.143:49184) 2024-01-16T15:00:35.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.427+0000 7fa50aa00640 1 -- 172.21.15.143:0/870803662 learned_addr learned my addr 172.21.15.143:0/870803662 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:35.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.427+0000 7fa50a000640 1 --2- 172.21.15.143:0/870803662 >> v2:172.21.15.204:3300/0 conn(0x7fa50c06e950 0x7fa50c161bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:35.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.427+0000 7fa509600640 1 -- 172.21.15.143:0/870803662 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 840394911 0 0) 0x7fa50c0ac840 con 0x7fa50c06b780 2024-01-16T15:00:35.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.427+0000 7fa509600640 1 -- 172.21.15.143:0/870803662 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4ec003520 con 0x7fa50c06b780 2024-01-16T15:00:35.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.427+0000 7fa509600640 1 -- 172.21.15.143:0/870803662 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3303315710 0 0) 0x7fa4ec003520 con 0x7fa50c06b780 2024-01-16T15:00:35.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.428+0000 7fa509600640 1 -- 172.21.15.143:0/870803662 >> v2:172.21.15.204:3300/0 conn(0x7fa50c06e950 msgr2=0x7fa50c161bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:35.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.428+0000 7fa509600640 1 --2- 172.21.15.143:0/870803662 >> v2:172.21.15.204:3300/0 conn(0x7fa50c06e950 0x7fa50c161bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:35.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.428+0000 7fa509600640 1 -- 172.21.15.143:0/870803662 >> v1:172.21.15.204:6789/0 conn(0x7fa50c06bb60 legacy=0x7fa50c0adb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:35.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.428+0000 7fa509600640 1 -- 172.21.15.143:0/870803662 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa50c0aca90 con 0x7fa50c06b780 2024-01-16T15:00:35.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.428+0000 7fa509600640 1 -- 172.21.15.143:0/870803662 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7fa4fc001670 con 0x7fa50c06b780 2024-01-16T15:00:35.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.428+0000 7fa509600640 1 -- 172.21.15.143:0/870803662 >> v1:172.21.15.143:6789/0 conn(0x7fa50c06b780 legacy=0x7fa50c0b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:35.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.428+0000 7fa509600640 1 --2- 172.21.15.143:0/870803662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4ec003970 0x7fa4ec003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:35.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.428+0000 7fa509600640 1 --2- 172.21.15.143:0/870803662 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4ec0047c0 0x7fa4ec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:35.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.428+0000 7fa509600640 1 --2- 172.21.15.143:0/870803662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4ec005180 0x7fa4ec00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:35.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.428+0000 7fa509600640 1 -- 172.21.15.143:0/870803662 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c162100 con 0x7fa4ec0047c0 2024-01-16T15:00:35.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.428+0000 7fa509600640 1 -- 172.21.15.143:0/870803662 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ec00c020 con 0x7fa4ec003970 2024-01-16T15:00:35.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.428+0000 7fa509600640 1 -- 172.21.15.143:0/870803662 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ec00c1e0 con 0x7fa4ec005180 2024-01-16T15:00:35.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.428+0000 7fa50a000640 1 --2- 172.21.15.143:0/870803662 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4ec0047c0 0x7fa4ec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:35.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.428+0000 7fa50b400640 1 --2- 172.21.15.143:0/870803662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4ec005180 0x7fa4ec00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:35.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.428+0000 7fa50aa00640 1 --2- 172.21.15.143:0/870803662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4ec003970 0x7fa4ec003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:35.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.429+0000 7fa50a000640 1 -- 172.21.15.143:0/870803662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4ec005180 msgr2=0x7fa4ec00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:35.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.429+0000 7fa50a000640 1 --2- 172.21.15.143:0/870803662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4ec005180 0x7fa4ec00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:35.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.429+0000 7fa50a000640 1 -- 172.21.15.143:0/870803662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4ec003970 msgr2=0x7fa4ec003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:35.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.429+0000 7fa50a000640 1 --2- 172.21.15.143:0/870803662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4ec003970 0x7fa4ec003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:35.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.429+0000 7fa50a000640 1 -- 172.21.15.143:0/870803662 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4ec003520 con 0x7fa4ec0047c0 2024-01-16T15:00:35.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.429+0000 7fa50b400640 1 --2- 172.21.15.143:0/870803662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4ec005180 0x7fa4ec00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:35.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.429+0000 7fa50aa00640 1 --2- 172.21.15.143:0/870803662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4ec003970 0x7fa4ec003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:35.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.429+0000 7fa50b400640 1 -- 172.21.15.143:0/870803662 reap_dead start 2024-01-16T15:00:35.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.430+0000 7fa50a000640 1 --2- 172.21.15.143:0/870803662 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4ec0047c0 0x7fa4ec004c40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4009f90 tx=0x7fa4ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93fe2b5b57aabe4c server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:35.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.430+0000 7fa509600640 1 -- 172.21.15.143:0/870803662 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f402cbd0 con 0x7fa4ec0047c0 2024-01-16T15:00:35.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.430+0000 7fa509600640 1 -- 172.21.15.143:0/870803662 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f400f040 con 0x7fa4ec0047c0 2024-01-16T15:00:35.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.430+0000 7fa509600640 1 -- 172.21.15.143:0/870803662 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f4034440 con 0x7fa4ec0047c0 2024-01-16T15:00:35.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.431+0000 7fa511e00640 1 -- 172.21.15.143:0/870803662 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4ec0047c0 msgr2=0x7fa4ec004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:35.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.431+0000 7fa511e00640 1 --2- 172.21.15.143:0/870803662 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4ec0047c0 0x7fa4ec004c40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4009f90 tx=0x7fa4ec0057f0 comp rx=0 tx=0).stop 2024-01-16T15:00:35.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.431+0000 7fa511e00640 1 -- 172.21.15.143:0/870803662 shutdown_connections 2024-01-16T15:00:35.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.431+0000 7fa511e00640 1 --2- 172.21.15.143:0/870803662 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4ec0047c0 0x7fa4ec004c40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:35.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.431+0000 7fa511e00640 1 -- 172.21.15.143:0/870803662 >> 172.21.15.143:0/870803662 conn(0x7fa50c070fa0 msgr2=0x7fa50c0713b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:35.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.431+0000 7fa511e00640 1 -- 172.21.15.143:0/870803662 shutdown_connections 2024-01-16T15:00:35.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.431+0000 7fa511e00640 1 -- 172.21.15.143:0/870803662 wait complete. 2024-01-16T15:00:35.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.433+0000 7fa511e00640 1 Processor -- start 2024-01-16T15:00:35.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.433+0000 7fa511e00640 1 -- start start 2024-01-16T15:00:35.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.434+0000 7fa511e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4ec0047c0 0x7fa50c1549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:35.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.434+0000 7fa511e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa50c154f30 0x7fa50c0b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:35.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.434+0000 7fa511e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa50c0b1160 0x7fa50c145040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:35.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.434+0000 7fa511e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c0b15f0 con 0x7fa50c0b1160 2024-01-16T15:00:35.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.434+0000 7fa511e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c153950 con 0x7fa50c154f30 2024-01-16T15:00:35.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.434+0000 7fa511e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c153c30 con 0x7fa4ec0047c0 2024-01-16T15:00:35.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.434+0000 7fa50b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa50c0b1160 0x7fa50c145040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:35.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.434+0000 7fa50b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa50c0b1160 0x7fa50c145040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43132/0 (socket says 172.21.15.143:43132) 2024-01-16T15:00:35.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.434+0000 7fa50b400640 1 -- 172.21.15.143:0/558442343 learned_addr learned my addr 172.21.15.143:0/558442343 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:35.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.434+0000 7fa50aa00640 1 --2- 172.21.15.143:0/558442343 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4ec0047c0 0x7fa50c1549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:35.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.435+0000 7fa50a000640 1 --2- 172.21.15.143:0/558442343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa50c154f30 0x7fa50c0b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:35.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.435+0000 7fa50b400640 1 -- 172.21.15.143:0/558442343 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4ec0047c0 msgr2=0x7fa50c1549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:35.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.435+0000 7fa50b400640 1 --2- 172.21.15.143:0/558442343 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4ec0047c0 0x7fa50c1549f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:35.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.435+0000 7fa50b400640 1 -- 172.21.15.143:0/558442343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa50c154f30 msgr2=0x7fa50c0b0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:35.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.435+0000 7fa50b400640 1 --2- 172.21.15.143:0/558442343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa50c154f30 0x7fa50c0b0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:35.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.435+0000 7fa50b400640 1 -- 172.21.15.143:0/558442343 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa50c153f10 con 0x7fa50c0b1160 2024-01-16T15:00:35.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.435+0000 7fa50a000640 1 --2- 172.21.15.143:0/558442343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa50c154f30 0x7fa50c0b0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:35.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.435+0000 7fa50b400640 1 --2- 172.21.15.143:0/558442343 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa50c0b1160 0x7fa50c145040 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa50c0af1b0 tx=0x7fa50400d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:35.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.435+0000 7fa503400640 1 -- 172.21.15.143:0/558442343 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50400b070 con 0x7fa50c0b1160 2024-01-16T15:00:35.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.435+0000 7fa503400640 1 -- 172.21.15.143:0/558442343 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5040054e0 con 0x7fa50c0b1160 2024-01-16T15:00:35.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.435+0000 7fa503400640 1 -- 172.21.15.143:0/558442343 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa504003830 con 0x7fa50c0b1160 2024-01-16T15:00:35.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.435+0000 7fa511e00640 1 -- 172.21.15.143:0/558442343 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa50c1541f0 con 0x7fa50c0b1160 2024-01-16T15:00:35.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.435+0000 7fa511e00640 1 -- 172.21.15.143:0/558442343 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa50c1513e0 con 0x7fa50c0b1160 2024-01-16T15:00:35.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.436+0000 7fa511e00640 1 -- 172.21.15.143:0/558442343 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa50c154710 con 0x7fa50c0b1160 2024-01-16T15:00:35.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.437+0000 7fa503400640 1 -- 172.21.15.143:0/558442343 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fa504003990 con 0x7fa50c0b1160 2024-01-16T15:00:35.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.438+0000 7fa503400640 1 --2- 172.21.15.143:0/558442343 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fa4e406b2f0 0x7fa4e406d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:35.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.438+0000 7fa503400640 1 -- 172.21.15.143:0/558442343 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa504088f10 con 0x7fa50c0b1160 2024-01-16T15:00:35.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.439+0000 7fa50aa00640 1 --2- 172.21.15.143:0/558442343 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fa4e406b2f0 0x7fa4e406d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:35.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.439+0000 7fa50aa00640 1 --2- 172.21.15.143:0/558442343 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fa4e406b2f0 0x7fa4e406d7b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc0017a0 tx=0x7fa4fc001470 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:35.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.442+0000 7fa503400640 1 -- 172.21.15.143:0/558442343 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa50c154710 con 0x7fa50c0b1160 2024-01-16T15:00:35.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.639+0000 7fa511e00640 1 -- 172.21.15.143:0/558442343 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa50c154710 con 0x7fa50c0b1160 2024-01-16T15:00:35.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.640+0000 7fa503400640 1 -- 172.21.15.143:0/558442343 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa50c154710 con 0x7fa50c0b1160 2024-01-16T15:00:35.621 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2024-01-16T15:00:35.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.643+0000 7fa511e00640 1 -- 172.21.15.143:0/558442343 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fa4e406b2f0 msgr2=0x7fa4e406d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:35.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.643+0000 7fa511e00640 1 --2- 172.21.15.143:0/558442343 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fa4e406b2f0 0x7fa4e406d7b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc0017a0 tx=0x7fa4fc001470 comp rx=0 tx=0).stop 2024-01-16T15:00:35.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.643+0000 7fa511e00640 1 -- 172.21.15.143:0/558442343 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa50c0b1160 msgr2=0x7fa50c145040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:35.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.643+0000 7fa511e00640 1 --2- 172.21.15.143:0/558442343 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa50c0b1160 0x7fa50c145040 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa50c0af1b0 tx=0x7fa50400d890 comp rx=0 tx=0).stop 2024-01-16T15:00:35.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.643+0000 7fa511e00640 1 -- 172.21.15.143:0/558442343 shutdown_connections 2024-01-16T15:00:35.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.643+0000 7fa511e00640 1 --2- 172.21.15.143:0/558442343 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fa4e406b2f0 0x7fa4e406d7b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:35.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.643+0000 7fa511e00640 1 --2- 172.21.15.143:0/558442343 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa50c0b1160 0x7fa50c145040 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:35.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.643+0000 7fa511e00640 1 --2- 172.21.15.143:0/558442343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa50c154f30 0x7fa50c0b0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:35.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.643+0000 7fa511e00640 1 --2- 172.21.15.143:0/558442343 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4ec0047c0 0x7fa50c1549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:35.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.643+0000 7fa511e00640 1 -- 172.21.15.143:0/558442343 >> 172.21.15.143:0/558442343 conn(0x7fa50c070fa0 msgr2=0x7fa50c0b5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:35.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.643+0000 7fa511e00640 1 -- 172.21.15.143:0/558442343 shutdown_connections 2024-01-16T15:00:35.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.644+0000 7fa511e00640 1 -- 172.21.15.143:0/558442343 wait complete. 2024-01-16T15:00:35.643 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-16T15:00:35.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.871+0000 7f0d39a00640 1 Processor -- start 2024-01-16T15:00:35.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.871+0000 7f0d39a00640 1 -- start start 2024-01-16T15:00:35.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.873+0000 7f0d39a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0d3415b2f0 0x7f0d340aef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:35.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.873+0000 7f0d39a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0d3415b6d0 0x7f0d3406b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:35.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.873+0000 7f0d39a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0d340b58a0 0x7f0d3406ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:35.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.873+0000 7f0d39a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d340b4eb0 con 0x7f0d3415b6d0 2024-01-16T15:00:35.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.873+0000 7f0d39a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d340b5620 con 0x7f0d3415b2f0 2024-01-16T15:00:35.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.873+0000 7f0d39a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d340cbf10 con 0x7f0d340b58a0 2024-01-16T15:00:35.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.873+0000 7f0d32000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0d3415b6d0 0x7f0d3406b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:35.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.873+0000 7f0d32000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0d3415b6d0 0x7f0d3406b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43146/0 (socket says 172.21.15.143:43146) 2024-01-16T15:00:35.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.873+0000 7f0d32000640 1 -- 172.21.15.143:0/3121556577 learned_addr learned my addr 172.21.15.143:0/3121556577 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:35.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.874+0000 7f0d32000640 1 -- 172.21.15.143:0/3121556577 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0d340b58a0 msgr2=0x7f0d3406ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:35.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.874+0000 7f0d32000640 1 --2- 172.21.15.143:0/3121556577 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0d340b58a0 0x7f0d3406ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:35.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.874+0000 7f0d32000640 1 -- 172.21.15.143:0/3121556577 >> v2:172.21.15.204:3300/0 conn(0x7f0d3415b2f0 msgr2=0x7f0d340aef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:35.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.874+0000 7f0d32a00640 1 --2- 172.21.15.143:0/3121556577 >> v2:172.21.15.204:3300/0 conn(0x7f0d3415b2f0 0x7f0d340aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:35.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.874+0000 7f0d32000640 1 --2- 172.21.15.143:0/3121556577 >> v2:172.21.15.204:3300/0 conn(0x7f0d3415b2f0 0x7f0d340aef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:35.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.874+0000 7f0d32000640 1 -- 172.21.15.143:0/3121556577 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d340cc1f0 con 0x7f0d3415b6d0 2024-01-16T15:00:35.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.875+0000 7f0d32000640 1 --2- 172.21.15.143:0/3121556577 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d3415b6d0 0x7f0d3406b510 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0d340cfa70 tx=0x7f0d1c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d9d8b366dc7e3c1 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:35.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.875+0000 7f0d31600640 1 -- 172.21.15.143:0/3121556577 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c02c9e0 con 0x7f0d3415b6d0 2024-01-16T15:00:35.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.875+0000 7f0d31600640 1 -- 172.21.15.143:0/3121556577 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1c02cb40 con 0x7f0d3415b6d0 2024-01-16T15:00:35.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.875+0000 7f0d31600640 1 -- 172.21.15.143:0/3121556577 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c0343f0 con 0x7f0d3415b6d0 2024-01-16T15:00:35.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.876+0000 7f0d39a00640 1 -- 172.21.15.143:0/3121556577 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d3415b6d0 msgr2=0x7f0d3406b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:35.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.876+0000 7f0d39a00640 1 --2- 172.21.15.143:0/3121556577 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d3415b6d0 0x7f0d3406b510 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0d340cfa70 tx=0x7f0d1c02be40 comp rx=0 tx=0).stop 2024-01-16T15:00:35.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.876+0000 7f0d39a00640 1 -- 172.21.15.143:0/3121556577 shutdown_connections 2024-01-16T15:00:35.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.876+0000 7f0d39a00640 1 --2- 172.21.15.143:0/3121556577 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0d340b58a0 0x7f0d3406ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:35.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.876+0000 7f0d39a00640 1 --2- 172.21.15.143:0/3121556577 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d3415b6d0 0x7f0d3406b510 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:35.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.876+0000 7f0d39a00640 1 --2- 172.21.15.143:0/3121556577 >> v2:172.21.15.204:3300/0 conn(0x7f0d3415b2f0 0x7f0d340aef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:35.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.876+0000 7f0d39a00640 1 -- 172.21.15.143:0/3121556577 >> 172.21.15.143:0/3121556577 conn(0x7f0d34070fa0 msgr2=0x7f0d340713b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:35.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.876+0000 7f0d39a00640 1 -- 172.21.15.143:0/3121556577 shutdown_connections 2024-01-16T15:00:35.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.876+0000 7f0d39a00640 1 -- 172.21.15.143:0/3121556577 wait complete. 2024-01-16T15:00:35.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.878+0000 7f0d39a00640 1 Processor -- start 2024-01-16T15:00:35.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.878+0000 7f0d39a00640 1 -- start start 2024-01-16T15:00:35.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.879+0000 7f0d39a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0d340b58a0 0x7f0d3414f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:35.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.879+0000 7f0d39a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d3415b2f0 0x7f0d341f11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:35.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.879+0000 7f0d39a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d3415b6d0 0x7f0d34142560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:35.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.879+0000 7f0d39a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3414e930 con 0x7f0d3415b2f0 2024-01-16T15:00:35.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.879+0000 7f0d39a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d34142aa0 con 0x7f0d3415b6d0 2024-01-16T15:00:35.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.879+0000 7f0d39a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3414f820 con 0x7f0d340b58a0 2024-01-16T15:00:35.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.879+0000 7f0d32000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d3415b2f0 0x7f0d341f11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:35.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.880+0000 7f0d32000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d3415b2f0 0x7f0d341f11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43150/0 (socket says 172.21.15.143:43150) 2024-01-16T15:00:35.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.880+0000 7f0d32000640 1 -- 172.21.15.143:0/3822585788 learned_addr learned my addr 172.21.15.143:0/3822585788 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:35.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.880+0000 7f0d32a00640 1 --2- 172.21.15.143:0/3822585788 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0d340b58a0 0x7f0d3414f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:35.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.880+0000 7f0d33400640 1 --2- 172.21.15.143:0/3822585788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d3415b6d0 0x7f0d34142560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:35.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.880+0000 7f0d32000640 1 -- 172.21.15.143:0/3822585788 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0d340b58a0 msgr2=0x7f0d3414f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:35.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.880+0000 7f0d32000640 1 --2- 172.21.15.143:0/3822585788 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0d340b58a0 0x7f0d3414f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:35.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.880+0000 7f0d32000640 1 -- 172.21.15.143:0/3822585788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d3415b6d0 msgr2=0x7f0d34142560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:35.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.880+0000 7f0d32000640 1 --2- 172.21.15.143:0/3822585788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d3415b6d0 0x7f0d34142560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:35.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.880+0000 7f0d32000640 1 -- 172.21.15.143:0/3822585788 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d1c009b30 con 0x7f0d3415b2f0 2024-01-16T15:00:35.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.880+0000 7f0d33400640 1 --2- 172.21.15.143:0/3822585788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d3415b6d0 0x7f0d34142560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:35.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.880+0000 7f0d32000640 1 --2- 172.21.15.143:0/3822585788 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d3415b2f0 0x7f0d341f11b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c000f80 tx=0x7f0d1c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:35.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.881+0000 7f0d2aa00640 1 -- 172.21.15.143:0/3822585788 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c03e450 con 0x7f0d3415b2f0 2024-01-16T15:00:35.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.881+0000 7f0d2aa00640 1 -- 172.21.15.143:0/3822585788 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1c03e5f0 con 0x7f0d3415b2f0 2024-01-16T15:00:35.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.881+0000 7f0d2aa00640 1 -- 172.21.15.143:0/3822585788 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c03e9a0 con 0x7f0d3415b2f0 2024-01-16T15:00:35.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.881+0000 7f0d39a00640 1 -- 172.21.15.143:0/3822585788 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d3415cdb0 con 0x7f0d3415b2f0 2024-01-16T15:00:35.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.881+0000 7f0d39a00640 1 -- 172.21.15.143:0/3822585788 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d3415d370 con 0x7f0d3415b2f0 2024-01-16T15:00:35.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.882+0000 7f0d39a00640 1 -- 172.21.15.143:0/3822585788 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d3404e8b0 con 0x7f0d3415b2f0 2024-01-16T15:00:35.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.883+0000 7f0d2aa00640 1 -- 172.21.15.143:0/3822585788 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f0d1c03eb00 con 0x7f0d3415b2f0 2024-01-16T15:00:35.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.884+0000 7f0d2aa00640 1 --2- 172.21.15.143:0/3822585788 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0d0c06b2c0 0x7f0d0c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:35.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.884+0000 7f0d2aa00640 1 -- 172.21.15.143:0/3822585788 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0d1c0accf0 con 0x7f0d3415b2f0 2024-01-16T15:00:35.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.884+0000 7f0d32a00640 1 --2- 172.21.15.143:0/3822585788 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0d0c06b2c0 0x7f0d0c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:35.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.885+0000 7f0d32a00640 1 --2- 172.21.15.143:0/3822585788 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0d0c06b2c0 0x7f0d0c06d780 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0d2400a970 tx=0x7f0d24008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:35.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:35.889+0000 7f0d2aa00640 1 -- 172.21.15.143:0/3822585788 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0d1c077be0 con 0x7f0d3415b2f0 2024-01-16T15:00:36.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.077+0000 7f0d39a00640 1 -- 172.21.15.143:0/3822585788 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0d3415ed40 con 0x7f0d3415b2f0 2024-01-16T15:00:36.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.078+0000 7f0d2aa00640 1 -- 172.21.15.143:0/3822585788 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0d1c03edb0 con 0x7f0d3415b2f0 2024-01-16T15:00:36.058 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2024-01-16T15:00:36.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.081+0000 7f0d39a00640 1 -- 172.21.15.143:0/3822585788 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0d0c06b2c0 msgr2=0x7f0d0c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:36.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.081+0000 7f0d39a00640 1 --2- 172.21.15.143:0/3822585788 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0d0c06b2c0 0x7f0d0c06d780 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0d2400a970 tx=0x7f0d24008040 comp rx=0 tx=0).stop 2024-01-16T15:00:36.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.081+0000 7f0d39a00640 1 -- 172.21.15.143:0/3822585788 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d3415b2f0 msgr2=0x7f0d341f11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:36.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.081+0000 7f0d39a00640 1 --2- 172.21.15.143:0/3822585788 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d3415b2f0 0x7f0d341f11b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c000f80 tx=0x7f0d1c004a40 comp rx=0 tx=0).stop 2024-01-16T15:00:36.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.081+0000 7f0d39a00640 1 -- 172.21.15.143:0/3822585788 shutdown_connections 2024-01-16T15:00:36.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.081+0000 7f0d39a00640 1 --2- 172.21.15.143:0/3822585788 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0d0c06b2c0 0x7f0d0c06d780 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:36.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.081+0000 7f0d39a00640 1 --2- 172.21.15.143:0/3822585788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d3415b6d0 0x7f0d34142560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:36.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.081+0000 7f0d39a00640 1 --2- 172.21.15.143:0/3822585788 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d3415b2f0 0x7f0d341f11b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:36.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.081+0000 7f0d39a00640 1 --2- 172.21.15.143:0/3822585788 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0d340b58a0 0x7f0d3414f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:36.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.081+0000 7f0d39a00640 1 -- 172.21.15.143:0/3822585788 >> 172.21.15.143:0/3822585788 conn(0x7f0d34070fa0 msgr2=0x7f0d341598f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:36.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.081+0000 7f0d39a00640 1 -- 172.21.15.143:0/3822585788 shutdown_connections 2024-01-16T15:00:36.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.081+0000 7f0d39a00640 1 -- 172.21.15.143:0/3822585788 wait complete. 2024-01-16T15:00:36.081 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T15:00:36.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.311+0000 7f75a2800640 1 Processor -- start 2024-01-16T15:00:36.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.311+0000 7f75a2800640 1 -- start start 2024-01-16T15:00:36.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.313+0000 7f75a2800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f759c159e80 0x7f759c071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:36.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.313+0000 7f75a2800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f759c0ce870 con 0x7f759c06d5c0 2024-01-16T15:00:36.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.313+0000 7f75a2800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f759c0cdc60 con 0x7f759c0cd6a0 2024-01-16T15:00:36.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.313+0000 7f75a2800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f759c0cdf40 con 0x7f759c159e80 2024-01-16T15:00:36.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.313+0000 7f75a1e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f759c06d5c0 0x7f759c163430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49198/0 (socket says 172.21.15.143:49198) 2024-01-16T15:00:36.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.313+0000 7f75a1e00640 1 -- 172.21.15.143:0/4264920484 learned_addr learned my addr 172.21.15.143:0/4264920484 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:36.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.314+0000 7f75a1400640 1 --2- 172.21.15.143:0/4264920484 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f759c159e80 0x7f759c071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:36.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.314+0000 7f759be00640 1 -- 172.21.15.143:0/4264920484 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3425699638 0 0) 0x7f759c0ce870 con 0x7f759c06d5c0 2024-01-16T15:00:36.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.314+0000 7f759be00640 1 -- 172.21.15.143:0/4264920484 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f757c005730 con 0x7f759c06d5c0 2024-01-16T15:00:36.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.314+0000 7f759be00640 1 -- 172.21.15.143:0/4264920484 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3341890202 0 0) 0x7f757c005730 con 0x7f759c06d5c0 2024-01-16T15:00:36.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.314+0000 7f759be00640 1 -- 172.21.15.143:0/4264920484 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f759c159e80 msgr2=0x7f759c071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:36.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.314+0000 7f759be00640 1 --2- 172.21.15.143:0/4264920484 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f759c159e80 0x7f759c071610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:36.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.314+0000 7f759be00640 1 -- 172.21.15.143:0/4264920484 >> v1:172.21.15.204:6789/0 conn(0x7f759c0cd6a0 legacy=0x7f759c15a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:36.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.314+0000 7f759be00640 1 -- 172.21.15.143:0/4264920484 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f759c0ce220 con 0x7f759c06d5c0 2024-01-16T15:00:36.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.314+0000 7f759be00640 1 -- 172.21.15.143:0/4264920484 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7f75940019f0 con 0x7f759c06d5c0 2024-01-16T15:00:36.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.314+0000 7f759be00640 1 -- 172.21.15.143:0/4264920484 >> v1:172.21.15.143:6789/0 conn(0x7f759c06d5c0 legacy=0x7f759c163430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:36.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.314+0000 7f759be00640 1 --2- 172.21.15.143:0/4264920484 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f757c005bb0 0x7f757c005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:36.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.314+0000 7f759be00640 1 --2- 172.21.15.143:0/4264920484 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f759c159e80 0x7f757c0069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:36.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.314+0000 7f759be00640 1 --2- 172.21.15.143:0/4264920484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f757c006f10 0x7f757c007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:36.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.315+0000 7f759be00640 1 -- 172.21.15.143:0/4264920484 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759c0cdf40 con 0x7f757c005bb0 2024-01-16T15:00:36.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.315+0000 7f759be00640 1 -- 172.21.15.143:0/4264920484 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757c00ff90 con 0x7f757c006f10 2024-01-16T15:00:36.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.315+0000 7f759be00640 1 -- 172.21.15.143:0/4264920484 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f757c0100d0 con 0x7f759c159e80 2024-01-16T15:00:36.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.315+0000 7f75a1400640 1 --2- 172.21.15.143:0/4264920484 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f757c005bb0 0x7f757c005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:36.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.315+0000 7f75a0a00640 1 --2- 172.21.15.143:0/4264920484 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f759c159e80 0x7f757c0069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:36.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.315+0000 7f75a1400640 1 -- 172.21.15.143:0/4264920484 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f759c159e80 msgr2=0x7f757c0069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:36.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.315+0000 7f75a1400640 1 --2- 172.21.15.143:0/4264920484 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f759c159e80 0x7f757c0069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:36.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.315+0000 7f75a1400640 1 -- 172.21.15.143:0/4264920484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f757c006f10 msgr2=0x7f757c007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:36.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.315+0000 7f75a1400640 1 --2- 172.21.15.143:0/4264920484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f757c006f10 0x7f757c007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:36.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.315+0000 7f75a1400640 1 -- 172.21.15.143:0/4264920484 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757c005730 con 0x7f757c005bb0 2024-01-16T15:00:36.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.315+0000 7f75a0a00640 1 --2- 172.21.15.143:0/4264920484 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f759c159e80 0x7f757c0069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:36.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.316+0000 7f75a1400640 1 --2- 172.21.15.143:0/4264920484 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f757c005bb0 0x7f757c005fd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f758c009ac0 tx=0x7f757c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae5dc4d3e2d62154 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:36.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.317+0000 7f759be00640 1 -- 172.21.15.143:0/4264920484 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758c02c6b0 con 0x7f757c005bb0 2024-01-16T15:00:36.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.317+0000 7f759be00640 1 -- 172.21.15.143:0/4264920484 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f758c02c810 con 0x7f757c005bb0 2024-01-16T15:00:36.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.317+0000 7f759be00640 1 -- 172.21.15.143:0/4264920484 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758c02cba0 con 0x7f757c005bb0 2024-01-16T15:00:36.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.317+0000 7f75a2800640 1 -- 172.21.15.143:0/4264920484 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f757c005bb0 msgr2=0x7f757c005fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:36.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.317+0000 7f75a2800640 1 --2- 172.21.15.143:0/4264920484 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f757c005bb0 0x7f757c005fd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f758c009ac0 tx=0x7f757c007a00 comp rx=0 tx=0).stop 2024-01-16T15:00:36.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.317+0000 7f75a1e00640 1 -- 172.21.15.143:0/4264920484 reap_dead start 2024-01-16T15:00:36.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.317+0000 7f75a2800640 1 -- 172.21.15.143:0/4264920484 shutdown_connections 2024-01-16T15:00:36.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.317+0000 7f75a2800640 1 -- 172.21.15.143:0/4264920484 >> 172.21.15.143:0/4264920484 conn(0x7f759c0cb830 msgr2=0x7f759c0cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:36.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.317+0000 7f75a2800640 1 -- 172.21.15.143:0/4264920484 shutdown_connections 2024-01-16T15:00:36.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.317+0000 7f75a2800640 1 -- 172.21.15.143:0/4264920484 wait complete. 2024-01-16T15:00:36.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.319+0000 7f75a2800640 1 Processor -- start 2024-01-16T15:00:36.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.319+0000 7f75a2800640 1 -- start start 2024-01-16T15:00:36.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.320+0000 7f75a2800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f757c005bb0 0x7f759c0b2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:36.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.320+0000 7f75a2800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f759c0b3e10 0x7f759c13fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:36.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.320+0000 7f75a2800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f759c140420 0x7f759c1edfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:36.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.320+0000 7f75a2800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759c1470e0 con 0x7f759c140420 2024-01-16T15:00:36.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.320+0000 7f75a2800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759c0b2d70 con 0x7f757c005bb0 2024-01-16T15:00:36.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.320+0000 7f75a2800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f759c0b3050 con 0x7f759c0b3e10 2024-01-16T15:00:36.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.320+0000 7f75a1e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f759c140420 0x7f759c1edfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:36.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.320+0000 7f75a1e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f759c140420 0x7f759c1edfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43176/0 (socket says 172.21.15.143:43176) 2024-01-16T15:00:36.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.320+0000 7f75a1e00640 1 -- 172.21.15.143:0/294975158 learned_addr learned my addr 172.21.15.143:0/294975158 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:36.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.321+0000 7f75a0a00640 1 --2- 172.21.15.143:0/294975158 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f759c0b3e10 0x7f759c13fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:36.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.321+0000 7f75a1400640 1 --2- 172.21.15.143:0/294975158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f757c005bb0 0x7f759c0b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:36.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.321+0000 7f75a1e00640 1 -- 172.21.15.143:0/294975158 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f759c0b3e10 msgr2=0x7f759c13fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:36.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.321+0000 7f75a1e00640 1 --2- 172.21.15.143:0/294975158 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f759c0b3e10 0x7f759c13fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:36.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.321+0000 7f75a1e00640 1 -- 172.21.15.143:0/294975158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f757c005bb0 msgr2=0x7f759c0b2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:36.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.321+0000 7f75a1e00640 1 --2- 172.21.15.143:0/294975158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f757c005bb0 0x7f759c0b2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:36.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.321+0000 7f75a1e00640 1 -- 172.21.15.143:0/294975158 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f759c0b3330 con 0x7f759c140420 2024-01-16T15:00:36.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.321+0000 7f75a1400640 1 --2- 172.21.15.143:0/294975158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f757c005bb0 0x7f759c0b2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:36.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.321+0000 7f75a1e00640 1 --2- 172.21.15.143:0/294975158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f759c140420 0x7f759c1edfd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f75940012e0 tx=0x7f759400cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:36.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.321+0000 7f7599600640 1 -- 172.21.15.143:0/294975158 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759400a610 con 0x7f759c140420 2024-01-16T15:00:36.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.321+0000 7f7599600640 1 -- 172.21.15.143:0/294975158 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f759400a770 con 0x7f759c140420 2024-01-16T15:00:36.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.321+0000 7f7599600640 1 -- 172.21.15.143:0/294975158 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759400ab20 con 0x7f759c140420 2024-01-16T15:00:36.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.321+0000 7f75a2800640 1 -- 172.21.15.143:0/294975158 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f759c0b3610 con 0x7f759c140420 2024-01-16T15:00:36.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.322+0000 7f75a2800640 1 -- 172.21.15.143:0/294975158 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f759c1408a0 con 0x7f759c140420 2024-01-16T15:00:36.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.323+0000 7f75a2800640 1 -- 172.21.15.143:0/294975158 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f759c0b3b30 con 0x7f759c140420 2024-01-16T15:00:36.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.324+0000 7f7599600640 1 -- 172.21.15.143:0/294975158 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f759400ac80 con 0x7f759c140420 2024-01-16T15:00:36.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.324+0000 7f7599600640 1 --2- 172.21.15.143:0/294975158 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f757406b2f0 0x7f757406d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:36.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.325+0000 7f7599600640 1 -- 172.21.15.143:0/294975158 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7594089450 con 0x7f759c140420 2024-01-16T15:00:36.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.325+0000 7f75a1400640 1 --2- 172.21.15.143:0/294975158 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f757406b2f0 0x7f757406d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:36.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.325+0000 7f75a1400640 1 --2- 172.21.15.143:0/294975158 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f757406b2f0 0x7f757406d7b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f758c036000 tx=0x7f758c030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:36.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.329+0000 7f7599600640 1 -- 172.21.15.143:0/294975158 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f759c0b3b30 con 0x7f759c140420 2024-01-16T15:00:36.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.519+0000 7f75a2800640 1 -- 172.21.15.143:0/294975158 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f759c0b3b30 con 0x7f759c140420 2024-01-16T15:00:36.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.521+0000 7f7599600640 1 -- 172.21.15.143:0/294975158 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16565 (secure 0 0 0) 0x7f759c0b3b30 con 0x7f759c140420 2024-01-16T15:00:36.502 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:00:36.502 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0","created":"2024-01-16T14:59:38.532601+0000","modified":"2024-01-16T15:00:04.543620+0000","last_up_change":"2024-01-16T14:59:50.983699+0000","last_in_change":"2024-01-16T14:59:42.056297+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:59:51.727720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T15:00:01.010472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T15:00:02.041228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f4c8891d-ab29-472a-907d-8446f20f7012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6801","nonce":1569293626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6805","nonce":1569293626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6814","nonce":1569293626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6809","nonce":1569293626}]},"public_addr":"172.21.15.143:6801/1569293626","cluster_addr":"172.21.15.143:6805/1569293626","heartbeat_back_addr":"172.21.15.143:6814/1569293626","heartbeat_front_addr":"172.21.15.143:6809/1569293626","state":["exists","up"]},{"osd":1,"uuid":"c4a2788d-8b89-4992-82b2-7f4b9debd509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6804","nonce":1276841168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6808","nonce":1276841168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6813","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6815","nonce":1276841168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6811","nonce":1276841168}]},"public_addr":"172.21.15.143:6804/1276841168","cluster_addr":"172.21.15.143:6808/1276841168","heartbeat_back_addr":"172.21.15.143:6815/1276841168","heartbeat_front_addr":"172.21.15.143:6811/1276841168","state":["exists","up"]},{"osd":2,"uuid":"64645c1d-31b2-4794-b0f2-7ccf8df30393","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6817","nonce":1217670170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6819","nonce":1217670170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6823","nonce":1217670170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6821","nonce":1217670170}]},"public_addr":"172.21.15.143:6817/1217670170","cluster_addr":"172.21.15.143:6819/1217670170","heartbeat_back_addr":"172.21.15.143:6823/1217670170","heartbeat_front_addr":"172.21.15.143:6821/1217670170","state":["exists","up"]},{"osd":3,"uuid":"73674129-a84e-4d31-8247-7d0179ae6df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6825","nonce":3414188012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6827","nonce":3414188012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6831","nonce":3414188012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6829","nonce":3414188012}]},"public_addr":"172.21.15.143:6825/3414188012","cluster_addr":"172.21.15.143:6827/3414188012","heartbeat_back_addr":"172.21.15.143:6831/3414188012","heartbeat_front_addr":"172.21.15.143:6829/3414188012","state":["exists","up"]},{"osd":4,"uuid":"cea28c9b-2f1c-4087-b415-9960789131d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6811","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6812","nonce":1070255716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6816","nonce":1070255716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6823","nonce":1070255716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6820","nonce":1070255716}]},"public_addr":"172.21.15.204:6812/1070255716","cluster_addr":"172.21.15.204:6816/1070255716","heartbeat_back_addr":"172.21.15.204:6823/1070255716","heartbeat_front_addr":"172.21.15.204:6820/1070255716","state":["exists","up"]},{"osd":5,"uuid":"33b63560-5574-4e78-983f-b398d67a2cb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6809","nonce":3605417837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6813","nonce":3605417837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6819","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6821","nonce":3605417837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6817","nonce":3605417837}]},"public_addr":"172.21.15.204:6809/3605417837","cluster_addr":"172.21.15.204:6813/3605417837","heartbeat_back_addr":"172.21.15.204:6821/3605417837","heartbeat_front_addr":"172.21.15.204:6817/3605417837","state":["exists","up"]},{"osd":6,"uuid":"df397427-8940-4eee-9c79-eece669d735d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6801","nonce":1085000534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6803","nonce":1085000534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6808","nonce":1085000534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6805","nonce":1085000534}]},"public_addr":"172.21.15.204:6801/1085000534","cluster_addr":"172.21.15.204:6803/1085000534","heartbeat_back_addr":"172.21.15.204:6808/1085000534","heartbeat_front_addr":"172.21.15.204:6805/1085000534","state":["exists","up"]},{"osd":7,"uuid":"a2e5a15c-8312-405a-ac02-f7e1fc9df2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6825","nonce":1337220609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6827","nonce":1337220609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6831","nonce":1337220609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6829","nonce":1337220609}]},"public_addr":"172.21.15.204:6825/1337220609","cluster_addr":"172.21.15.204:6827/1337220609","heartbeat_back_addr":"172.21.15.204:6831/1337220609","heartbeat_front_addr":"172.21.15.204:6829/1337220609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:49.984507+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.395306+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.632998+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.373627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.342920+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.438366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.331926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.406657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T15:00:36.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.524+0000 7f75a2800640 1 -- 172.21.15.143:0/294975158 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f757406b2f0 msgr2=0x7f757406d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:36.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.524+0000 7f75a2800640 1 --2- 172.21.15.143:0/294975158 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f757406b2f0 0x7f757406d7b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f758c036000 tx=0x7f758c030000 comp rx=0 tx=0).stop 2024-01-16T15:00:36.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.524+0000 7f75a2800640 1 -- 172.21.15.143:0/294975158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f759c140420 msgr2=0x7f759c1edfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:36.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.524+0000 7f75a2800640 1 --2- 172.21.15.143:0/294975158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f759c140420 0x7f759c1edfd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f75940012e0 tx=0x7f759400cf00 comp rx=0 tx=0).stop 2024-01-16T15:00:36.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.524+0000 7f75a2800640 1 -- 172.21.15.143:0/294975158 shutdown_connections 2024-01-16T15:00:36.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.524+0000 7f75a2800640 1 --2- 172.21.15.143:0/294975158 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f757406b2f0 0x7f757406d7b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:36.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.524+0000 7f75a2800640 1 --2- 172.21.15.143:0/294975158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f759c140420 0x7f759c1edfd0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:36.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.524+0000 7f75a2800640 1 --2- 172.21.15.143:0/294975158 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f759c0b3e10 0x7f759c13fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:36.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.524+0000 7f75a2800640 1 --2- 172.21.15.143:0/294975158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f757c005bb0 0x7f759c0b2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:36.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.524+0000 7f75a2800640 1 -- 172.21.15.143:0/294975158 >> 172.21.15.143:0/294975158 conn(0x7f759c0cb830 msgr2=0x7f759c0b0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:36.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.524+0000 7f75a2800640 1 -- 172.21.15.143:0/294975158 shutdown_connections 2024-01-16T15:00:36.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.525+0000 7f75a2800640 1 -- 172.21.15.143:0/294975158 wait complete. 2024-01-16T15:00:36.523 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-16T14:59:51.727720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-16T15:00:01.010472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-16T15:00:02.041228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-16T15:00:36.523 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-16T15:00:36.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.751+0000 7fa3af800640 1 Processor -- start 2024-01-16T15:00:36.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.751+0000 7fa3af800640 1 -- start start 2024-01-16T15:00:36.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.752+0000 7fa3af800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa3a8072160 0x7fa3a815e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:36.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.753+0000 7fa3af800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa3a806de10 0x7fa3a80705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:36.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.753+0000 7fa3af800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3a815ec90 con 0x7fa3a80c49f0 2024-01-16T15:00:36.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.753+0000 7fa3af800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3a80cd530 con 0x7fa3a806de10 2024-01-16T15:00:36.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.753+0000 7fa3af800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3a80c4ed0 con 0x7fa3a8072160 2024-01-16T15:00:36.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.753+0000 7fa3ad800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa3a806de10 0x7fa3a80705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:36.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.753+0000 7fa39fe00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fa3a80c49f0 0x7fa3a806d700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49214/0 (socket says 172.21.15.143:49214) 2024-01-16T15:00:36.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.753+0000 7fa39fe00640 1 -- 172.21.15.143:0/578874410 learned_addr learned my addr 172.21.15.143:0/578874410 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:36.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.753+0000 7fa3ad800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa3a806de10 0x7fa3a80705c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43186/0 (socket says 172.21.15.143:43186) 2024-01-16T15:00:36.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.753+0000 7fa3ace00640 1 --2- 172.21.15.143:0/578874410 >> v2:172.21.15.204:3300/0 conn(0x7fa3a8072160 0x7fa3a815e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:36.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.753+0000 7fa3ad800640 1 -- 172.21.15.143:0/578874410 >> v2:172.21.15.204:3300/0 conn(0x7fa3a8072160 msgr2=0x7fa3a815e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:36.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.753+0000 7fa3ad800640 1 --2- 172.21.15.143:0/578874410 >> v2:172.21.15.204:3300/0 conn(0x7fa3a8072160 0x7fa3a815e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:36.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.754+0000 7fa3ad800640 1 -- 172.21.15.143:0/578874410 >> v1:172.21.15.143:6789/0 conn(0x7fa3a80c49f0 legacy=0x7fa3a806d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:36.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.754+0000 7fa3ad800640 1 -- 172.21.15.143:0/578874410 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a80c51b0 con 0x7fa3a806de10 2024-01-16T15:00:36.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.754+0000 7fa3a7e00640 1 -- 172.21.15.143:0/578874410 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4213253292 0 0) 0x7fa390001110 con 0x7fa3a80c49f0 2024-01-16T15:00:36.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.755+0000 7fa3ad800640 1 --2- 172.21.15.143:0/578874410 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3a806de10 0x7fa3a80705c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa3a80ccae0 tx=0x7fa3a002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4129d0053b89f9e2 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:36.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.755+0000 7fa3a7e00640 1 -- 172.21.15.143:0/578874410 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a002cc80 con 0x7fa3a806de10 2024-01-16T15:00:36.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.755+0000 7fa3a7e00640 1 -- 172.21.15.143:0/578874410 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a002cde0 con 0x7fa3a806de10 2024-01-16T15:00:36.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.755+0000 7fa3a7e00640 1 -- 172.21.15.143:0/578874410 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0034660 con 0x7fa3a806de10 2024-01-16T15:00:36.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.756+0000 7fa3af800640 1 -- 172.21.15.143:0/578874410 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3a806de10 msgr2=0x7fa3a80705c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:36.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.756+0000 7fa3af800640 1 --2- 172.21.15.143:0/578874410 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3a806de10 0x7fa3a80705c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa3a80ccae0 tx=0x7fa3a002bed0 comp rx=0 tx=0).stop 2024-01-16T15:00:36.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.756+0000 7fa3af800640 1 -- 172.21.15.143:0/578874410 shutdown_connections 2024-01-16T15:00:36.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.756+0000 7fa3af800640 1 --2- 172.21.15.143:0/578874410 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3a806de10 0x7fa3a80705c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:36.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.756+0000 7fa3af800640 1 --2- 172.21.15.143:0/578874410 >> v2:172.21.15.204:3300/0 conn(0x7fa3a8072160 0x7fa3a815e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:36.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.756+0000 7fa3af800640 1 -- 172.21.15.143:0/578874410 >> 172.21.15.143:0/578874410 conn(0x7fa3a80b0c50 msgr2=0x7fa3a80b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:36.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.756+0000 7fa3af800640 1 -- 172.21.15.143:0/578874410 shutdown_connections 2024-01-16T15:00:36.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.756+0000 7fa3af800640 1 -- 172.21.15.143:0/578874410 wait complete. 2024-01-16T15:00:36.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.758+0000 7fa3af800640 1 Processor -- start 2024-01-16T15:00:36.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.758+0000 7fa3af800640 1 -- start start 2024-01-16T15:00:36.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.759+0000 7fa3af800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3a806de10 0x7fa3a809a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:36.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.759+0000 7fa3af800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3a8072160 0x7fa3a80999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:36.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.759+0000 7fa3af800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa3a80c49f0 0x7fa3a814fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:36.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.759+0000 7fa3af800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a80b26e0 con 0x7fa3a8072160 2024-01-16T15:00:36.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.759+0000 7fa3af800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a80b1db0 con 0x7fa3a806de10 2024-01-16T15:00:36.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.759+0000 7fa3af800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a80b2090 con 0x7fa3a80c49f0 2024-01-16T15:00:36.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.759+0000 7fa39fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3a8072160 0x7fa3a80999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:36.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.759+0000 7fa39fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3a8072160 0x7fa3a80999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43198/0 (socket says 172.21.15.143:43198) 2024-01-16T15:00:36.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.759+0000 7fa39fe00640 1 -- 172.21.15.143:0/2792766307 learned_addr learned my addr 172.21.15.143:0/2792766307 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:36.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.759+0000 7fa39fe00640 1 -- 172.21.15.143:0/2792766307 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa3a80c49f0 msgr2=0x7fa3a814fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:36.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.759+0000 7fa39fe00640 1 --2- 172.21.15.143:0/2792766307 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa3a80c49f0 0x7fa3a814fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:36.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.759+0000 7fa39fe00640 1 -- 172.21.15.143:0/2792766307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3a806de10 msgr2=0x7fa3a809a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:36.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.759+0000 7fa39fe00640 1 --2- 172.21.15.143:0/2792766307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3a806de10 0x7fa3a809a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:36.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.759+0000 7fa39fe00640 1 -- 172.21.15.143:0/2792766307 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a0009a70 con 0x7fa3a8072160 2024-01-16T15:00:36.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.760+0000 7fa39fe00640 1 --2- 172.21.15.143:0/2792766307 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3a8072160 0x7fa3a80999d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa39000c430 tx=0x7fa39000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:36.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.760+0000 7fa3a5600640 1 -- 172.21.15.143:0/2792766307 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3900074e0 con 0x7fa3a8072160 2024-01-16T15:00:36.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.760+0000 7fa3a5600640 1 -- 172.21.15.143:0/2792766307 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa390007660 con 0x7fa3a8072160 2024-01-16T15:00:36.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.760+0000 7fa3af800640 1 -- 172.21.15.143:0/2792766307 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3a80b2370 con 0x7fa3a8072160 2024-01-16T15:00:36.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.760+0000 7fa3a5600640 1 -- 172.21.15.143:0/2792766307 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa390004a20 con 0x7fa3a8072160 2024-01-16T15:00:36.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.760+0000 7fa3af800640 1 -- 172.21.15.143:0/2792766307 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3a809a920 con 0x7fa3a8072160 2024-01-16T15:00:36.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.761+0000 7fa3af800640 1 -- 172.21.15.143:0/2792766307 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3a804e8b0 con 0x7fa3a8072160 2024-01-16T15:00:36.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.762+0000 7fa3a5600640 1 -- 172.21.15.143:0/2792766307 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fa390004b80 con 0x7fa3a8072160 2024-01-16T15:00:36.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.763+0000 7fa3a5600640 1 --2- 172.21.15.143:0/2792766307 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fa38006b2c0 0x7fa38006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:36.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.763+0000 7fa3a5600640 1 -- 172.21.15.143:0/2792766307 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa390089840 con 0x7fa3a8072160 2024-01-16T15:00:36.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.763+0000 7fa3ace00640 1 --2- 172.21.15.143:0/2792766307 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fa38006b2c0 0x7fa38006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:36.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.764+0000 7fa3ace00640 1 --2- 172.21.15.143:0/2792766307 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fa38006b2c0 0x7fa38006d780 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa3980098f0 tx=0x7fa398008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:36.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.768+0000 7fa3a5600640 1 -- 172.21.15.143:0/2792766307 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa3900546b0 con 0x7fa3a8072160 2024-01-16T15:00:36.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.956+0000 7fa3af800640 1 -- 172.21.15.143:0/2792766307 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa3a815ec90 con 0x7fa3a8072160 2024-01-16T15:00:36.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.957+0000 7fa3a5600640 1 -- 172.21.15.143:0/2792766307 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa3a815ec90 con 0x7fa3a8072160 2024-01-16T15:00:36.937 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-16T15:00:36.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.960+0000 7fa3af800640 1 -- 172.21.15.143:0/2792766307 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fa38006b2c0 msgr2=0x7fa38006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:36.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.960+0000 7fa3af800640 1 --2- 172.21.15.143:0/2792766307 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fa38006b2c0 0x7fa38006d780 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa3980098f0 tx=0x7fa398008040 comp rx=0 tx=0).stop 2024-01-16T15:00:36.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.960+0000 7fa3af800640 1 -- 172.21.15.143:0/2792766307 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3a8072160 msgr2=0x7fa3a80999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:36.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.960+0000 7fa3af800640 1 --2- 172.21.15.143:0/2792766307 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3a8072160 0x7fa3a80999d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa39000c430 tx=0x7fa39000c8f0 comp rx=0 tx=0).stop 2024-01-16T15:00:36.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.960+0000 7fa3af800640 1 -- 172.21.15.143:0/2792766307 shutdown_connections 2024-01-16T15:00:36.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.960+0000 7fa3af800640 1 --2- 172.21.15.143:0/2792766307 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fa38006b2c0 0x7fa38006d780 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:36.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.960+0000 7fa3af800640 1 --2- 172.21.15.143:0/2792766307 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa3a80c49f0 0x7fa3a814fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:36.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.960+0000 7fa3af800640 1 --2- 172.21.15.143:0/2792766307 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3a8072160 0x7fa3a80999d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:36.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.960+0000 7fa3af800640 1 --2- 172.21.15.143:0/2792766307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3a806de10 0x7fa3a809a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:36.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.960+0000 7fa3af800640 1 -- 172.21.15.143:0/2792766307 >> 172.21.15.143:0/2792766307 conn(0x7fa3a80b0c50 msgr2=0x7fa3a80b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:36.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.960+0000 7fa3af800640 1 -- 172.21.15.143:0/2792766307 shutdown_connections 2024-01-16T15:00:36.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:36.960+0000 7fa3af800640 1 -- 172.21.15.143:0/2792766307 wait complete. 2024-01-16T15:00:36.959 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-16T15:00:37.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.188+0000 7f036b000640 1 Processor -- start 2024-01-16T15:00:37.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.189+0000 7f036b000640 1 -- start start 2024-01-16T15:00:37.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.190+0000 7f036b000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f036406e160 0x7f03640ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:37.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.190+0000 7f036b000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f03640cd110 0x7f0364070e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:37.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.190+0000 7f036b000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0364161a10 con 0x7f0364146eb0 2024-01-16T15:00:37.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.190+0000 7f036b000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03640cd580 con 0x7f03640cd110 2024-01-16T15:00:37.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.190+0000 7f036b000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0364161cf0 con 0x7f036406e160 2024-01-16T15:00:37.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.191+0000 7f0363e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f0364146eb0 0x7f0364161150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49220/0 (socket says 172.21.15.143:49220) 2024-01-16T15:00:37.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.191+0000 7f0363e00640 1 -- 172.21.15.143:0/2689942226 learned_addr learned my addr 172.21.15.143:0/2689942226 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:37.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.191+0000 7f0369000640 1 --2- 172.21.15.143:0/2689942226 >> v2:172.21.15.143:3300/0 conn(0x7f03640cd110 0x7f0364070e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:37.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.191+0000 7f0363400640 1 --2- 172.21.15.143:0/2689942226 >> v2:172.21.15.204:3300/0 conn(0x7f036406e160 0x7f03640ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:37.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.191+0000 7f0362a00640 1 -- 172.21.15.143:0/2689942226 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 584560644 0 0) 0x7f0364161a10 con 0x7f0364146eb0 2024-01-16T15:00:37.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.191+0000 7f0362a00640 1 -- 172.21.15.143:0/2689942226 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0344003520 con 0x7f0364146eb0 2024-01-16T15:00:37.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.191+0000 7f0369000640 1 -- 172.21.15.143:0/2689942226 >> v2:172.21.15.204:3300/0 conn(0x7f036406e160 msgr2=0x7f03640ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:37.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.191+0000 7f0369000640 1 --2- 172.21.15.143:0/2689942226 >> v2:172.21.15.204:3300/0 conn(0x7f036406e160 0x7f03640ccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:37.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.191+0000 7f0369000640 1 -- 172.21.15.143:0/2689942226 >> v1:172.21.15.143:6789/0 conn(0x7f0364146eb0 legacy=0x7f0364161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:37.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.191+0000 7f0369000640 1 -- 172.21.15.143:0/2689942226 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0364161fd0 con 0x7f03640cd110 2024-01-16T15:00:37.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.192+0000 7f0369000640 1 --2- 172.21.15.143:0/2689942226 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03640cd110 0x7f0364070e40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f036406eab0 tx=0x7f035c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f829aa230e4edd server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:37.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.193+0000 7f0362a00640 1 -- 172.21.15.143:0/2689942226 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035c031830 con 0x7f03640cd110 2024-01-16T15:00:37.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.193+0000 7f0362a00640 1 -- 172.21.15.143:0/2689942226 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f035c031990 con 0x7f03640cd110 2024-01-16T15:00:37.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.193+0000 7f0362a00640 1 -- 172.21.15.143:0/2689942226 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035c031d00 con 0x7f03640cd110 2024-01-16T15:00:37.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.193+0000 7f036b000640 1 -- 172.21.15.143:0/2689942226 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03640cd110 msgr2=0x7f0364070e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:37.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.193+0000 7f036b000640 1 --2- 172.21.15.143:0/2689942226 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03640cd110 0x7f0364070e40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f036406eab0 tx=0x7f035c02b8d0 comp rx=0 tx=0).stop 2024-01-16T15:00:37.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.193+0000 7f036b000640 1 -- 172.21.15.143:0/2689942226 shutdown_connections 2024-01-16T15:00:37.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.194+0000 7f036b000640 1 --2- 172.21.15.143:0/2689942226 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03640cd110 0x7f0364070e40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:37.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.194+0000 7f036b000640 1 --2- 172.21.15.143:0/2689942226 >> v2:172.21.15.204:3300/0 conn(0x7f036406e160 0x7f03640ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:37.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.194+0000 7f036b000640 1 -- 172.21.15.143:0/2689942226 >> 172.21.15.143:0/2689942226 conn(0x7f03640b0c50 msgr2=0x7f03640b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:37.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.194+0000 7f036b000640 1 -- 172.21.15.143:0/2689942226 shutdown_connections 2024-01-16T15:00:37.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.194+0000 7f036b000640 1 -- 172.21.15.143:0/2689942226 wait complete. 2024-01-16T15:00:37.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.195+0000 7f036b000640 1 Processor -- start 2024-01-16T15:00:37.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.195+0000 7f036b000640 1 -- start start 2024-01-16T15:00:37.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.196+0000 7f036b000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f036406e160 0x7f036409cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:37.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.196+0000 7f036b000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03640cd110 0x7f03641437b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:37.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.196+0000 7f036b000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0364146eb0 0x7f03641f3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:37.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.196+0000 7f036b000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03641f45a0 con 0x7f036406e160 2024-01-16T15:00:37.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.196+0000 7f036b000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0364160910 con 0x7f03640cd110 2024-01-16T15:00:37.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.196+0000 7f036b000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03641f4250 con 0x7f0364146eb0 2024-01-16T15:00:37.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.197+0000 7f0363e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f036406e160 0x7f036409cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:37.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.197+0000 7f0363e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f036406e160 0x7f036409cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43216/0 (socket says 172.21.15.143:43216) 2024-01-16T15:00:37.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.197+0000 7f0363e00640 1 -- 172.21.15.143:0/634562145 learned_addr learned my addr 172.21.15.143:0/634562145 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:37.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.197+0000 7f0363e00640 1 -- 172.21.15.143:0/634562145 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0364146eb0 msgr2=0x7f03641f3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:37.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.197+0000 7f0363e00640 1 --2- 172.21.15.143:0/634562145 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0364146eb0 0x7f03641f3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:37.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.197+0000 7f0363e00640 1 -- 172.21.15.143:0/634562145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03640cd110 msgr2=0x7f03641437b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:37.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.197+0000 7f0363e00640 1 --2- 172.21.15.143:0/634562145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03640cd110 0x7f03641437b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:37.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.197+0000 7f0363e00640 1 -- 172.21.15.143:0/634562145 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f035c009400 con 0x7f036406e160 2024-01-16T15:00:37.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.197+0000 7f0363e00640 1 --2- 172.21.15.143:0/634562145 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f036406e160 0x7f036409cd50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f035400caa0 tx=0x7f035400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:37.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.198+0000 7f035be00640 1 -- 172.21.15.143:0/634562145 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035400ac00 con 0x7f036406e160 2024-01-16T15:00:37.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.198+0000 7f035be00640 1 -- 172.21.15.143:0/634562145 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f035400ad60 con 0x7f036406e160 2024-01-16T15:00:37.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.198+0000 7f035be00640 1 -- 172.21.15.143:0/634562145 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0354012620 con 0x7f036406e160 2024-01-16T15:00:37.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.198+0000 7f036b000640 1 -- 172.21.15.143:0/634562145 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03641f3910 con 0x7f036406e160 2024-01-16T15:00:37.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.198+0000 7f036b000640 1 -- 172.21.15.143:0/634562145 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03641f3ed0 con 0x7f036406e160 2024-01-16T15:00:37.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.199+0000 7f036b000640 1 -- 172.21.15.143:0/634562145 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f036404e8b0 con 0x7f036406e160 2024-01-16T15:00:37.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.200+0000 7f035be00640 1 -- 172.21.15.143:0/634562145 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f0354007070 con 0x7f036406e160 2024-01-16T15:00:37.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.201+0000 7f035be00640 1 --2- 172.21.15.143:0/634562145 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f033c06b2c0 0x7f033c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:37.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.201+0000 7f035be00640 1 -- 172.21.15.143:0/634562145 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0354089b10 con 0x7f036406e160 2024-01-16T15:00:37.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.201+0000 7f0363400640 1 --2- 172.21.15.143:0/634562145 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f033c06b2c0 0x7f033c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:37.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.202+0000 7f0363400640 1 --2- 172.21.15.143:0/634562145 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f033c06b2c0 0x7f033c06d780 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f036415ec30 tx=0x7f034c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:37.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.206+0000 7f035be00640 1 -- 172.21.15.143:0/634562145 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0354054a10 con 0x7f036406e160 2024-01-16T15:00:37.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.396+0000 7f036b000640 1 -- 172.21.15.143:0/634562145 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f036406ded0 con 0x7f036406e160 2024-01-16T15:00:37.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.397+0000 7f035be00640 1 -- 172.21.15.143:0/634562145 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0354017020 con 0x7f036406e160 2024-01-16T15:00:37.377 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2024-01-16T15:00:37.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.399+0000 7f036b000640 1 -- 172.21.15.143:0/634562145 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f033c06b2c0 msgr2=0x7f033c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:37.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.399+0000 7f036b000640 1 --2- 172.21.15.143:0/634562145 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f033c06b2c0 0x7f033c06d780 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f036415ec30 tx=0x7f034c008040 comp rx=0 tx=0).stop 2024-01-16T15:00:37.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.399+0000 7f036b000640 1 -- 172.21.15.143:0/634562145 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f036406e160 msgr2=0x7f036409cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:37.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.399+0000 7f036b000640 1 --2- 172.21.15.143:0/634562145 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f036406e160 0x7f036409cd50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f035400caa0 tx=0x7f035400cf60 comp rx=0 tx=0).stop 2024-01-16T15:00:37.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.400+0000 7f036b000640 1 -- 172.21.15.143:0/634562145 shutdown_connections 2024-01-16T15:00:37.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.400+0000 7f036b000640 1 --2- 172.21.15.143:0/634562145 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f033c06b2c0 0x7f033c06d780 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:37.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.400+0000 7f036b000640 1 --2- 172.21.15.143:0/634562145 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0364146eb0 0x7f03641f3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:37.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.400+0000 7f036b000640 1 --2- 172.21.15.143:0/634562145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03640cd110 0x7f03641437b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:37.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.400+0000 7f036b000640 1 --2- 172.21.15.143:0/634562145 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f036406e160 0x7f036409cd50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:37.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.400+0000 7f036b000640 1 -- 172.21.15.143:0/634562145 >> 172.21.15.143:0/634562145 conn(0x7f03640b0c50 msgr2=0x7f0364145aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:37.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.400+0000 7f036b000640 1 -- 172.21.15.143:0/634562145 shutdown_connections 2024-01-16T15:00:37.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.400+0000 7f036b000640 1 -- 172.21.15.143:0/634562145 wait complete. 2024-01-16T15:00:37.398 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-16T15:00:37.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.630+0000 7fa764800640 1 Processor -- start 2024-01-16T15:00:37.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.630+0000 7fa764800640 1 -- start start 2024-01-16T15:00:37.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.632+0000 7fa764800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa75c0cc0f0 0x7fa75c0c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:37.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.632+0000 7fa764800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa75c15eea0 con 0x7fa75c15b440 2024-01-16T15:00:37.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.632+0000 7fa764800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa75c0abf30 con 0x7fa75c0c57e0 2024-01-16T15:00:37.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.632+0000 7fa764800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa75c15b820 con 0x7fa75c0cc0f0 2024-01-16T15:00:37.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.632+0000 7fa761400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa75c0cc0f0 0x7fa75c0c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:37.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.632+0000 7fa761e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fa75c15b440 0x7fa75c0cb9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49236/0 (socket says 172.21.15.143:49236) 2024-01-16T15:00:37.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.632+0000 7fa761e00640 1 -- 172.21.15.143:0/3533216492 learned_addr learned my addr 172.21.15.143:0/3533216492 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:37.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.633+0000 7fa761400640 1 -- 172.21.15.143:0/3533216492 >> v1:172.21.15.204:6789/0 conn(0x7fa75c0c57e0 legacy=0x7fa75c06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:37.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.633+0000 7fa761400640 1 -- 172.21.15.143:0/3533216492 >> v1:172.21.15.143:6789/0 conn(0x7fa75c15b440 legacy=0x7fa75c0cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:37.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.633+0000 7fa761400640 1 -- 172.21.15.143:0/3533216492 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa75c15f0f0 con 0x7fa75c0cc0f0 2024-01-16T15:00:37.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.634+0000 7fa761400640 1 --2- 172.21.15.143:0/3533216492 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa75c0cc0f0 0x7fa75c0c52a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa7440097a0 tx=0x7fa74402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3232a24e6bb1ccfb server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:37.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.634+0000 7fa760a00640 1 -- 172.21.15.143:0/3533216492 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74402e070 con 0x7fa75c0cc0f0 2024-01-16T15:00:37.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.634+0000 7fa760a00640 1 -- 172.21.15.143:0/3533216492 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7440315a0 con 0x7fa75c0cc0f0 2024-01-16T15:00:37.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.634+0000 7fa760a00640 1 -- 172.21.15.143:0/3533216492 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa744031930 con 0x7fa75c0cc0f0 2024-01-16T15:00:37.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.635+0000 7fa764800640 1 -- 172.21.15.143:0/3533216492 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa75c0cc0f0 msgr2=0x7fa75c0c52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:37.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.635+0000 7fa764800640 1 --2- 172.21.15.143:0/3533216492 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa75c0cc0f0 0x7fa75c0c52a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa7440097a0 tx=0x7fa74402b840 comp rx=0 tx=0).stop 2024-01-16T15:00:37.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.635+0000 7fa764800640 1 -- 172.21.15.143:0/3533216492 shutdown_connections 2024-01-16T15:00:37.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.635+0000 7fa764800640 1 --2- 172.21.15.143:0/3533216492 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa75c0cc0f0 0x7fa75c0c52a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:37.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.635+0000 7fa764800640 1 -- 172.21.15.143:0/3533216492 >> 172.21.15.143:0/3533216492 conn(0x7fa75c0b0c50 msgr2=0x7fa75c0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:37.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.635+0000 7fa764800640 1 -- 172.21.15.143:0/3533216492 shutdown_connections 2024-01-16T15:00:37.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.635+0000 7fa764800640 1 -- 172.21.15.143:0/3533216492 wait complete. 2024-01-16T15:00:37.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.637+0000 7fa764800640 1 Processor -- start 2024-01-16T15:00:37.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.637+0000 7fa764800640 1 -- start start 2024-01-16T15:00:37.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.638+0000 7fa764800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa75c0c57e0 0x7fa75c159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:37.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.638+0000 7fa764800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa75c0cc0f0 0x7fa75c1f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:37.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.638+0000 7fa764800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa75c15b440 0x7fa75c0704c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:37.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.638+0000 7fa764800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c1566e0 con 0x7fa75c0c57e0 2024-01-16T15:00:37.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.638+0000 7fa764800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c1557f0 con 0x7fa75c15b440 2024-01-16T15:00:37.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.638+0000 7fa764800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c155ad0 con 0x7fa75c0cc0f0 2024-01-16T15:00:37.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.638+0000 7fa761e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa75c0c57e0 0x7fa75c159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:37.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.638+0000 7fa761e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa75c0c57e0 0x7fa75c159820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43222/0 (socket says 172.21.15.143:43222) 2024-01-16T15:00:37.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.638+0000 7fa761e00640 1 -- 172.21.15.143:0/3501358553 learned_addr learned my addr 172.21.15.143:0/3501358553 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:37.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.638+0000 7fa761e00640 1 -- 172.21.15.143:0/3501358553 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa75c0cc0f0 msgr2=0x7fa75c1f3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:37.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.638+0000 7fa761e00640 1 --2- 172.21.15.143:0/3501358553 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa75c0cc0f0 0x7fa75c1f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:37.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.638+0000 7fa761e00640 1 -- 172.21.15.143:0/3501358553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa75c15b440 msgr2=0x7fa75c0704c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:37.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.638+0000 7fa761e00640 1 --2- 172.21.15.143:0/3501358553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa75c15b440 0x7fa75c0704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:37.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.638+0000 7fa761e00640 1 -- 172.21.15.143:0/3501358553 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa744009400 con 0x7fa75c0c57e0 2024-01-16T15:00:37.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.639+0000 7fa761e00640 1 --2- 172.21.15.143:0/3501358553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa75c0c57e0 0x7fa75c159820 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa74c00cbc0 tx=0x7fa74c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:37.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.639+0000 7fa75a000640 1 -- 172.21.15.143:0/3501358553 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74c00aca0 con 0x7fa75c0c57e0 2024-01-16T15:00:37.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.639+0000 7fa75a000640 1 -- 172.21.15.143:0/3501358553 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa74c00ae00 con 0x7fa75c0c57e0 2024-01-16T15:00:37.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.639+0000 7fa75a000640 1 -- 172.21.15.143:0/3501358553 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74c012600 con 0x7fa75c0c57e0 2024-01-16T15:00:37.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.639+0000 7fa764800640 1 -- 172.21.15.143:0/3501358553 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa75c155db0 con 0x7fa75c0c57e0 2024-01-16T15:00:37.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.639+0000 7fa764800640 1 -- 172.21.15.143:0/3501358553 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa75c156370 con 0x7fa75c0c57e0 2024-01-16T15:00:37.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.640+0000 7fa764800640 1 -- 172.21.15.143:0/3501358553 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa75c04e8b0 con 0x7fa75c0c57e0 2024-01-16T15:00:37.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.641+0000 7fa75a000640 1 -- 172.21.15.143:0/3501358553 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fa74c007070 con 0x7fa75c0c57e0 2024-01-16T15:00:37.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.642+0000 7fa75a000640 1 --2- 172.21.15.143:0/3501358553 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fa73406b2c0 0x7fa73406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:37.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.642+0000 7fa75a000640 1 -- 172.21.15.143:0/3501358553 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa74c089aa0 con 0x7fa75c0c57e0 2024-01-16T15:00:37.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.642+0000 7fa761400640 1 --2- 172.21.15.143:0/3501358553 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fa73406b2c0 0x7fa73406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:37.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.643+0000 7fa761400640 1 --2- 172.21.15.143:0/3501358553 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fa73406b2c0 0x7fa73406d780 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa744036000 tx=0x7fa744030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:37.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.646+0000 7fa75a000640 1 -- 172.21.15.143:0/3501358553 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa74c0549b0 con 0x7fa75c0c57e0 2024-01-16T15:00:37.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.834+0000 7fa764800640 1 -- 172.21.15.143:0/3501358553 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa75c06dc80 con 0x7fa75c0c57e0 2024-01-16T15:00:37.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.835+0000 7fa75a000640 1 -- 172.21.15.143:0/3501358553 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa74c017020 con 0x7fa75c0c57e0 2024-01-16T15:00:37.815 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2024-01-16T15:00:37.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.838+0000 7fa764800640 1 -- 172.21.15.143:0/3501358553 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fa73406b2c0 msgr2=0x7fa73406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:37.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.838+0000 7fa764800640 1 --2- 172.21.15.143:0/3501358553 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fa73406b2c0 0x7fa73406d780 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa744036000 tx=0x7fa744030000 comp rx=0 tx=0).stop 2024-01-16T15:00:37.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.838+0000 7fa764800640 1 -- 172.21.15.143:0/3501358553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa75c0c57e0 msgr2=0x7fa75c159820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:37.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.838+0000 7fa764800640 1 --2- 172.21.15.143:0/3501358553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa75c0c57e0 0x7fa75c159820 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa74c00cbc0 tx=0x7fa74c00a060 comp rx=0 tx=0).stop 2024-01-16T15:00:37.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.838+0000 7fa764800640 1 -- 172.21.15.143:0/3501358553 shutdown_connections 2024-01-16T15:00:37.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.838+0000 7fa764800640 1 --2- 172.21.15.143:0/3501358553 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fa73406b2c0 0x7fa73406d780 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:37.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.838+0000 7fa764800640 1 --2- 172.21.15.143:0/3501358553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa75c15b440 0x7fa75c0704c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:37.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.838+0000 7fa764800640 1 --2- 172.21.15.143:0/3501358553 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa75c0cc0f0 0x7fa75c1f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:37.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.838+0000 7fa764800640 1 --2- 172.21.15.143:0/3501358553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa75c0c57e0 0x7fa75c159820 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:37.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.838+0000 7fa764800640 1 -- 172.21.15.143:0/3501358553 >> 172.21.15.143:0/3501358553 conn(0x7fa75c0b0c50 msgr2=0x7fa75c071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:37.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.838+0000 7fa764800640 1 -- 172.21.15.143:0/3501358553 shutdown_connections 2024-01-16T15:00:37.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:37.838+0000 7fa764800640 1 -- 172.21.15.143:0/3501358553 wait complete. 2024-01-16T15:00:37.835 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T15:00:38.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.056+0000 7fde9c200640 1 Processor -- start 2024-01-16T15:00:38.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.056+0000 7fde9c200640 1 -- start start 2024-01-16T15:00:38.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.058+0000 7fde9c200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fde940cf450 0x7fde940cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:38.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.058+0000 7fde9c200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde940cece0 con 0x7fde940cdfb0 2024-01-16T15:00:38.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.058+0000 7fde9c200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde9406d040 con 0x7fde9415c7f0 2024-01-16T15:00:38.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.058+0000 7fde9c200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde940cfe40 con 0x7fde940cf450 2024-01-16T15:00:38.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.058+0000 7fde98e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fde940cf450 0x7fde940cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:38.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.058+0000 7fde98e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fde940cf450 0x7fde940cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43230/0 (socket says 172.21.15.143:43230) 2024-01-16T15:00:38.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.058+0000 7fde98e00640 1 -- 172.21.15.143:0/1665966340 learned_addr learned my addr 172.21.15.143:0/1665966340 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:38.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.058+0000 7fde9a200640 1 --1- 172.21.15.143:0/1665966340 >> v1:172.21.15.143:6789/0 conn(0x7fde940cdfb0 0x7fde940ce3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49238/0 (socket says 172.21.15.143:49238) 2024-01-16T15:00:38.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.058+0000 7fde98e00640 1 -- 172.21.15.143:0/1665966340 >> v1:172.21.15.204:6789/0 conn(0x7fde9415c7f0 legacy=0x7fde9406db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:38.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.058+0000 7fde98e00640 1 -- 172.21.15.143:0/1665966340 >> v1:172.21.15.143:6789/0 conn(0x7fde940cdfb0 legacy=0x7fde940ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:38.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.059+0000 7fde98e00640 1 -- 172.21.15.143:0/1665966340 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde9406d320 con 0x7fde940cf450 2024-01-16T15:00:38.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.058+0000 7fde93e00640 1 -- 172.21.15.143:0/1665966340 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 393664070 0 0) 0x7fde940cece0 con 0x7fde940cdfb0 2024-01-16T15:00:38.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.060+0000 7fde98e00640 1 --2- 172.21.15.143:0/1665966340 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fde940cf450 0x7fde940cf900 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fde7c009870 tx=0x7fde7c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d80f78455f07c2b4 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:38.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.060+0000 7fde93e00640 1 -- 172.21.15.143:0/1665966340 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7c030860 con 0x7fde940cf450 2024-01-16T15:00:38.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.060+0000 7fde93e00640 1 -- 172.21.15.143:0/1665966340 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde7c0309c0 con 0x7fde940cf450 2024-01-16T15:00:38.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.060+0000 7fde93e00640 1 -- 172.21.15.143:0/1665966340 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7c030d50 con 0x7fde940cf450 2024-01-16T15:00:38.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.060+0000 7fde9c200640 1 -- 172.21.15.143:0/1665966340 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fde940cf450 msgr2=0x7fde940cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:38.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.060+0000 7fde9c200640 1 --2- 172.21.15.143:0/1665966340 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fde940cf450 0x7fde940cf900 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fde7c009870 tx=0x7fde7c02b9a0 comp rx=0 tx=0).stop 2024-01-16T15:00:38.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.060+0000 7fde9c200640 1 -- 172.21.15.143:0/1665966340 shutdown_connections 2024-01-16T15:00:38.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.060+0000 7fde9c200640 1 --2- 172.21.15.143:0/1665966340 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fde940cf450 0x7fde940cf900 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:38.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.060+0000 7fde9c200640 1 -- 172.21.15.143:0/1665966340 >> 172.21.15.143:0/1665966340 conn(0x7fde9406a920 msgr2=0x7fde9406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:38.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.061+0000 7fde9c200640 1 -- 172.21.15.143:0/1665966340 shutdown_connections 2024-01-16T15:00:38.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.061+0000 7fde9c200640 1 -- 172.21.15.143:0/1665966340 wait complete. 2024-01-16T15:00:38.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.062+0000 7fde9c200640 1 Processor -- start 2024-01-16T15:00:38.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.062+0000 7fde9c200640 1 -- start start 2024-01-16T15:00:38.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.063+0000 7fde9c200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fde940cdfb0 0x7fde9414d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:38.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.063+0000 7fde9c200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fde940cf450 0x7fde940ad140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:38.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.063+0000 7fde9c200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde9415c7f0 0x7fde940ad680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:38.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.063+0000 7fde9c200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde941f68e0 con 0x7fde940cf450 2024-01-16T15:00:38.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.063+0000 7fde9c200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde941f6bc0 con 0x7fde9415c7f0 2024-01-16T15:00:38.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.063+0000 7fde9c200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde941f6ea0 con 0x7fde940cdfb0 2024-01-16T15:00:38.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.064+0000 7fde98e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fde940cf450 0x7fde940ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:38.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.064+0000 7fde98e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fde940cf450 0x7fde940ad140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43240/0 (socket says 172.21.15.143:43240) 2024-01-16T15:00:38.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.064+0000 7fde98e00640 1 -- 172.21.15.143:0/3367572924 learned_addr learned my addr 172.21.15.143:0/3367572924 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:38.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.064+0000 7fde99800640 1 --2- 172.21.15.143:0/3367572924 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fde940cdfb0 0x7fde9414d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:38.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.064+0000 7fde98e00640 1 -- 172.21.15.143:0/3367572924 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fde940cdfb0 msgr2=0x7fde9414d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:38.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.064+0000 7fde98e00640 1 --2- 172.21.15.143:0/3367572924 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fde940cdfb0 0x7fde9414d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:38.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.064+0000 7fde9a200640 1 --2- 172.21.15.143:0/3367572924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde9415c7f0 0x7fde940ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:38.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.064+0000 7fde98e00640 1 -- 172.21.15.143:0/3367572924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde9415c7f0 msgr2=0x7fde940ad680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:38.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.064+0000 7fde98e00640 1 --2- 172.21.15.143:0/3367572924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde9415c7f0 0x7fde940ad680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:38.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.064+0000 7fde98e00640 1 -- 172.21.15.143:0/3367572924 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde7c0094d0 con 0x7fde940cf450 2024-01-16T15:00:38.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.064+0000 7fde98e00640 1 --2- 172.21.15.143:0/3367572924 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fde940cf450 0x7fde940ad140 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fde7c009870 tx=0x7fde7c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:38.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.065+0000 7fde91600640 1 -- 172.21.15.143:0/3367572924 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7c039d60 con 0x7fde940cf450 2024-01-16T15:00:38.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.065+0000 7fde91600640 1 -- 172.21.15.143:0/3367572924 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde7c043400 con 0x7fde940cf450 2024-01-16T15:00:38.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.065+0000 7fde91600640 1 -- 172.21.15.143:0/3367572924 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7c0437b0 con 0x7fde940cf450 2024-01-16T15:00:38.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.065+0000 7fde9c200640 1 -- 172.21.15.143:0/3367572924 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde941f7180 con 0x7fde940cf450 2024-01-16T15:00:38.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.065+0000 7fde9c200640 1 -- 172.21.15.143:0/3367572924 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde940b4000 con 0x7fde940cf450 2024-01-16T15:00:38.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.066+0000 7fde9c200640 1 -- 172.21.15.143:0/3367572924 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde94067470 con 0x7fde940cf450 2024-01-16T15:00:38.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.067+0000 7fde91600640 1 -- 172.21.15.143:0/3367572924 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fde7c043910 con 0x7fde940cf450 2024-01-16T15:00:38.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.068+0000 7fde91600640 1 --2- 172.21.15.143:0/3367572924 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fde6c06b2c0 0x7fde6c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:38.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.068+0000 7fde91600640 1 -- 172.21.15.143:0/3367572924 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde7c03e070 con 0x7fde940cf450 2024-01-16T15:00:38.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.068+0000 7fde99800640 1 --2- 172.21.15.143:0/3367572924 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fde6c06b2c0 0x7fde6c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:38.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.069+0000 7fde99800640 1 --2- 172.21.15.143:0/3367572924 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fde6c06b2c0 0x7fde6c06d780 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fde8400a750 tx=0x7fde84008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:38.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.072+0000 7fde91600640 1 -- 172.21.15.143:0/3367572924 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fde7c077e70 con 0x7fde940cf450 2024-01-16T15:00:38.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.265+0000 7fde9c200640 1 -- 172.21.15.143:0/3367572924 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fde940cd730 con 0x7fde940cf450 2024-01-16T15:00:38.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.268+0000 7fde91600640 1 -- 172.21.15.143:0/3367572924 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16565 (secure 0 0 0) 0x7fde7c07b490 con 0x7fde940cf450 2024-01-16T15:00:38.248 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:00:38.248 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0","created":"2024-01-16T14:59:38.532601+0000","modified":"2024-01-16T15:00:04.543620+0000","last_up_change":"2024-01-16T14:59:50.983699+0000","last_in_change":"2024-01-16T14:59:42.056297+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:59:51.727720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T15:00:01.010472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T15:00:02.041228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f4c8891d-ab29-472a-907d-8446f20f7012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6801","nonce":1569293626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6805","nonce":1569293626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6814","nonce":1569293626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6809","nonce":1569293626}]},"public_addr":"172.21.15.143:6801/1569293626","cluster_addr":"172.21.15.143:6805/1569293626","heartbeat_back_addr":"172.21.15.143:6814/1569293626","heartbeat_front_addr":"172.21.15.143:6809/1569293626","state":["exists","up"]},{"osd":1,"uuid":"c4a2788d-8b89-4992-82b2-7f4b9debd509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6804","nonce":1276841168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6808","nonce":1276841168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6813","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6815","nonce":1276841168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6811","nonce":1276841168}]},"public_addr":"172.21.15.143:6804/1276841168","cluster_addr":"172.21.15.143:6808/1276841168","heartbeat_back_addr":"172.21.15.143:6815/1276841168","heartbeat_front_addr":"172.21.15.143:6811/1276841168","state":["exists","up"]},{"osd":2,"uuid":"64645c1d-31b2-4794-b0f2-7ccf8df30393","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6817","nonce":1217670170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6819","nonce":1217670170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6823","nonce":1217670170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6821","nonce":1217670170}]},"public_addr":"172.21.15.143:6817/1217670170","cluster_addr":"172.21.15.143:6819/1217670170","heartbeat_back_addr":"172.21.15.143:6823/1217670170","heartbeat_front_addr":"172.21.15.143:6821/1217670170","state":["exists","up"]},{"osd":3,"uuid":"73674129-a84e-4d31-8247-7d0179ae6df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6825","nonce":3414188012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6827","nonce":3414188012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6831","nonce":3414188012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6829","nonce":3414188012}]},"public_addr":"172.21.15.143:6825/3414188012","cluster_addr":"172.21.15.143:6827/3414188012","heartbeat_back_addr":"172.21.15.143:6831/3414188012","heartbeat_front_addr":"172.21.15.143:6829/3414188012","state":["exists","up"]},{"osd":4,"uuid":"cea28c9b-2f1c-4087-b415-9960789131d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6811","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6812","nonce":1070255716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6816","nonce":1070255716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6823","nonce":1070255716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6820","nonce":1070255716}]},"public_addr":"172.21.15.204:6812/1070255716","cluster_addr":"172.21.15.204:6816/1070255716","heartbeat_back_addr":"172.21.15.204:6823/1070255716","heartbeat_front_addr":"172.21.15.204:6820/1070255716","state":["exists","up"]},{"osd":5,"uuid":"33b63560-5574-4e78-983f-b398d67a2cb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6809","nonce":3605417837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6813","nonce":3605417837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6819","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6821","nonce":3605417837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6817","nonce":3605417837}]},"public_addr":"172.21.15.204:6809/3605417837","cluster_addr":"172.21.15.204:6813/3605417837","heartbeat_back_addr":"172.21.15.204:6821/3605417837","heartbeat_front_addr":"172.21.15.204:6817/3605417837","state":["exists","up"]},{"osd":6,"uuid":"df397427-8940-4eee-9c79-eece669d735d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6801","nonce":1085000534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6803","nonce":1085000534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6808","nonce":1085000534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6805","nonce":1085000534}]},"public_addr":"172.21.15.204:6801/1085000534","cluster_addr":"172.21.15.204:6803/1085000534","heartbeat_back_addr":"172.21.15.204:6808/1085000534","heartbeat_front_addr":"172.21.15.204:6805/1085000534","state":["exists","up"]},{"osd":7,"uuid":"a2e5a15c-8312-405a-ac02-f7e1fc9df2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6825","nonce":1337220609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6827","nonce":1337220609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6831","nonce":1337220609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6829","nonce":1337220609}]},"public_addr":"172.21.15.204:6825/1337220609","cluster_addr":"172.21.15.204:6827/1337220609","heartbeat_back_addr":"172.21.15.204:6831/1337220609","heartbeat_front_addr":"172.21.15.204:6829/1337220609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:49.984507+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.395306+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.632998+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.373627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.342920+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.438366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.331926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.406657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T15:00:38.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.270+0000 7fde9c200640 1 -- 172.21.15.143:0/3367572924 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fde6c06b2c0 msgr2=0x7fde6c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:38.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.270+0000 7fde9c200640 1 --2- 172.21.15.143:0/3367572924 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fde6c06b2c0 0x7fde6c06d780 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fde8400a750 tx=0x7fde84008040 comp rx=0 tx=0).stop 2024-01-16T15:00:38.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.270+0000 7fde9c200640 1 -- 172.21.15.143:0/3367572924 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fde940cf450 msgr2=0x7fde940ad140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:38.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.270+0000 7fde9c200640 1 --2- 172.21.15.143:0/3367572924 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fde940cf450 0x7fde940ad140 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fde7c009870 tx=0x7fde7c004730 comp rx=0 tx=0).stop 2024-01-16T15:00:38.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.270+0000 7fde9c200640 1 -- 172.21.15.143:0/3367572924 shutdown_connections 2024-01-16T15:00:38.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.270+0000 7fde9c200640 1 --2- 172.21.15.143:0/3367572924 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fde6c06b2c0 0x7fde6c06d780 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:38.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.270+0000 7fde9c200640 1 --2- 172.21.15.143:0/3367572924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde9415c7f0 0x7fde940ad680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:38.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.270+0000 7fde9c200640 1 --2- 172.21.15.143:0/3367572924 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fde940cf450 0x7fde940ad140 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:38.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.270+0000 7fde9c200640 1 --2- 172.21.15.143:0/3367572924 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fde940cdfb0 0x7fde9414d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:38.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.270+0000 7fde9c200640 1 -- 172.21.15.143:0/3367572924 >> 172.21.15.143:0/3367572924 conn(0x7fde9406a920 msgr2=0x7fde94154970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:38.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.270+0000 7fde9c200640 1 -- 172.21.15.143:0/3367572924 shutdown_connections 2024-01-16T15:00:38.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.270+0000 7fde9c200640 1 -- 172.21.15.143:0/3367572924 wait complete. 2024-01-16T15:00:38.269 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-16T14:59:51.727720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-16T15:00:01.010472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-16T15:00:02.041228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-16T15:00:38.269 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-16T15:00:38.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.495+0000 7fd003400640 1 Processor -- start 2024-01-16T15:00:38.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.496+0000 7fd003400640 1 -- start start 2024-01-16T15:00:38.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.497+0000 7fd003400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcffc146eb0 0x7fcffc161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:38.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.497+0000 7fd003400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcffc06e160 0x7fcffc0ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:38.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.497+0000 7fd003400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcffc0cd510 con 0x7fcffc0cd110 2024-01-16T15:00:38.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.497+0000 7fd003400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcffc161a10 con 0x7fcffc146eb0 2024-01-16T15:00:38.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.497+0000 7fd003400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcffc161cf0 con 0x7fcffc06e160 2024-01-16T15:00:38.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.497+0000 7fd000a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcffc146eb0 0x7fcffc161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:38.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.498+0000 7fd000a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcffc146eb0 0x7fcffc161150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43256/0 (socket says 172.21.15.143:43256) 2024-01-16T15:00:38.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.498+0000 7fd000a00640 1 -- 172.21.15.143:0/1544354779 learned_addr learned my addr 172.21.15.143:0/1544354779 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:38.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.498+0000 7fcff3c00640 1 --2- 172.21.15.143:0/1544354779 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcffc06e160 0x7fcffc0ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:38.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.498+0000 7fd000a00640 1 -- 172.21.15.143:0/1544354779 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcffc06e160 msgr2=0x7fcffc0ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:38.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.498+0000 7fd000a00640 1 --2- 172.21.15.143:0/1544354779 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcffc06e160 0x7fcffc0ccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:38.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.498+0000 7fd000a00640 1 -- 172.21.15.143:0/1544354779 >> v1:172.21.15.143:6789/0 conn(0x7fcffc0cd110 legacy=0x7fcffc06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:38.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.498+0000 7fd000a00640 1 -- 172.21.15.143:0/1544354779 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcffc161fd0 con 0x7fcffc146eb0 2024-01-16T15:00:38.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.498+0000 7fcffbe00640 1 -- 172.21.15.143:0/1544354779 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4188343571 0 0) 0x7fcffc0cd510 con 0x7fcffc0cd110 2024-01-16T15:00:38.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.499+0000 7fd000a00640 1 --2- 172.21.15.143:0/1544354779 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcffc146eb0 0x7fcffc161150 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcfec009870 tx=0x7fcfec02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d3ef3dceb35c1b server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:38.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.500+0000 7fcffbe00640 1 -- 172.21.15.143:0/1544354779 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfec030840 con 0x7fcffc146eb0 2024-01-16T15:00:38.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.500+0000 7fcffbe00640 1 -- 172.21.15.143:0/1544354779 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfec0309a0 con 0x7fcffc146eb0 2024-01-16T15:00:38.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.500+0000 7fcffbe00640 1 -- 172.21.15.143:0/1544354779 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfec030d30 con 0x7fcffc146eb0 2024-01-16T15:00:38.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.500+0000 7fd003400640 1 -- 172.21.15.143:0/1544354779 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcffc146eb0 msgr2=0x7fcffc161150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:38.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.500+0000 7fd003400640 1 --2- 172.21.15.143:0/1544354779 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcffc146eb0 0x7fcffc161150 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcfec009870 tx=0x7fcfec02b980 comp rx=0 tx=0).stop 2024-01-16T15:00:38.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.500+0000 7fd003400640 1 -- 172.21.15.143:0/1544354779 shutdown_connections 2024-01-16T15:00:38.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.500+0000 7fd003400640 1 --2- 172.21.15.143:0/1544354779 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcffc06e160 0x7fcffc0ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:38.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.500+0000 7fd003400640 1 --2- 172.21.15.143:0/1544354779 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcffc146eb0 0x7fcffc161150 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:38.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.500+0000 7fd003400640 1 -- 172.21.15.143:0/1544354779 >> 172.21.15.143:0/1544354779 conn(0x7fcffc0b0c50 msgr2=0x7fcffc0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:38.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.500+0000 7fd003400640 1 -- 172.21.15.143:0/1544354779 shutdown_connections 2024-01-16T15:00:38.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.500+0000 7fd003400640 1 -- 172.21.15.143:0/1544354779 wait complete. 2024-01-16T15:00:38.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.502+0000 7fd003400640 1 Processor -- start 2024-01-16T15:00:38.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.502+0000 7fd003400640 1 -- start start 2024-01-16T15:00:38.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.503+0000 7fd003400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcffc06e160 0x7fcffc141b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:38.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.503+0000 7fd003400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcffc0cd110 0x7fcffc1f2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:38.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.503+0000 7fd003400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcffc146eb0 0x7fcffc0acf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:38.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.503+0000 7fd003400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcffc15f770 con 0x7fcffc146eb0 2024-01-16T15:00:38.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.503+0000 7fd003400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcffc1421e0 con 0x7fcffc06e160 2024-01-16T15:00:38.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.503+0000 7fd003400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcffc1424c0 con 0x7fcffc0cd110 2024-01-16T15:00:38.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.504+0000 7fd001400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcffc146eb0 0x7fcffc0acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:38.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.504+0000 7fd001400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcffc146eb0 0x7fcffc0acf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43260/0 (socket says 172.21.15.143:43260) 2024-01-16T15:00:38.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.504+0000 7fd001400640 1 -- 172.21.15.143:0/1174383967 learned_addr learned my addr 172.21.15.143:0/1174383967 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:38.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.504+0000 7fd001400640 1 -- 172.21.15.143:0/1174383967 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcffc0cd110 msgr2=0x7fcffc1f2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:38.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.504+0000 7fcff3c00640 1 --2- 172.21.15.143:0/1174383967 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcffc0cd110 0x7fcffc1f2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:38.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.504+0000 7fd001400640 1 --2- 172.21.15.143:0/1174383967 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcffc0cd110 0x7fcffc1f2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:38.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.504+0000 7fd001400640 1 -- 172.21.15.143:0/1174383967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcffc06e160 msgr2=0x7fcffc141b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:38.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.504+0000 7fd001400640 1 --2- 172.21.15.143:0/1174383967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcffc06e160 0x7fcffc141b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:38.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.504+0000 7fd001400640 1 -- 172.21.15.143:0/1174383967 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfec0094d0 con 0x7fcffc146eb0 2024-01-16T15:00:38.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.504+0000 7fd001400640 1 --2- 172.21.15.143:0/1174383967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcffc146eb0 0x7fcffc0acf80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcff4001560 tx=0x7fcff400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:38.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.504+0000 7fcff9600640 1 -- 172.21.15.143:0/1174383967 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff4003da0 con 0x7fcffc146eb0 2024-01-16T15:00:38.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.505+0000 7fcff9600640 1 -- 172.21.15.143:0/1174383967 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff4005070 con 0x7fcffc146eb0 2024-01-16T15:00:38.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.505+0000 7fcff9600640 1 -- 172.21.15.143:0/1174383967 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff4005420 con 0x7fcffc146eb0 2024-01-16T15:00:38.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.505+0000 7fd003400640 1 -- 172.21.15.143:0/1174383967 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcffc1427a0 con 0x7fcffc146eb0 2024-01-16T15:00:38.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.505+0000 7fd003400640 1 -- 172.21.15.143:0/1174383967 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcffc0b2c10 con 0x7fcffc146eb0 2024-01-16T15:00:38.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.506+0000 7fd003400640 1 -- 172.21.15.143:0/1174383967 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcffc0b2a70 con 0x7fcffc146eb0 2024-01-16T15:00:38.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.507+0000 7fcff9600640 1 -- 172.21.15.143:0/1174383967 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fcff4005580 con 0x7fcffc146eb0 2024-01-16T15:00:38.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.507+0000 7fcff9600640 1 --2- 172.21.15.143:0/1174383967 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fcfd406b2f0 0x7fcfd406d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:38.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.508+0000 7fcff9600640 1 -- 172.21.15.143:0/1174383967 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcff4089490 con 0x7fcffc146eb0 2024-01-16T15:00:38.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.508+0000 7fd000a00640 1 --2- 172.21.15.143:0/1174383967 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fcfd406b2f0 0x7fcfd406d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:38.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.508+0000 7fd000a00640 1 --2- 172.21.15.143:0/1174383967 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fcfd406b2f0 0x7fcfd406d7b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcfec036000 tx=0x7fcfec03b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:38.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.512+0000 7fcff9600640 1 -- 172.21.15.143:0/1174383967 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcffc0b2a70 con 0x7fcffc146eb0 2024-01-16T15:00:38.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.704+0000 7fd003400640 1 -- 172.21.15.143:0/1174383967 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcffc0b2a70 con 0x7fcffc146eb0 2024-01-16T15:00:38.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.705+0000 7fcff9600640 1 -- 172.21.15.143:0/1174383967 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcff4017020 con 0x7fcffc146eb0 2024-01-16T15:00:38.685 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-16T15:00:38.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.707+0000 7fd003400640 1 -- 172.21.15.143:0/1174383967 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fcfd406b2f0 msgr2=0x7fcfd406d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:38.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.707+0000 7fd003400640 1 --2- 172.21.15.143:0/1174383967 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fcfd406b2f0 0x7fcfd406d7b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcfec036000 tx=0x7fcfec03b040 comp rx=0 tx=0).stop 2024-01-16T15:00:38.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.707+0000 7fd003400640 1 -- 172.21.15.143:0/1174383967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcffc146eb0 msgr2=0x7fcffc0acf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:38.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.707+0000 7fd003400640 1 --2- 172.21.15.143:0/1174383967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcffc146eb0 0x7fcffc0acf80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcff4001560 tx=0x7fcff400cc70 comp rx=0 tx=0).stop 2024-01-16T15:00:38.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.707+0000 7fd003400640 1 -- 172.21.15.143:0/1174383967 shutdown_connections 2024-01-16T15:00:38.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.707+0000 7fd003400640 1 --2- 172.21.15.143:0/1174383967 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fcfd406b2f0 0x7fcfd406d7b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:38.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.707+0000 7fd003400640 1 --2- 172.21.15.143:0/1174383967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcffc146eb0 0x7fcffc0acf80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:38.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.707+0000 7fd003400640 1 --2- 172.21.15.143:0/1174383967 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcffc0cd110 0x7fcffc1f2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:38.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.707+0000 7fd003400640 1 --2- 172.21.15.143:0/1174383967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcffc06e160 0x7fcffc141b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:38.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.707+0000 7fd003400640 1 -- 172.21.15.143:0/1174383967 >> 172.21.15.143:0/1174383967 conn(0x7fcffc0b0c50 msgr2=0x7fcffc0aee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:38.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.707+0000 7fd003400640 1 -- 172.21.15.143:0/1174383967 shutdown_connections 2024-01-16T15:00:38.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.707+0000 7fd003400640 1 -- 172.21.15.143:0/1174383967 wait complete. 2024-01-16T15:00:38.705 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-16T15:00:38.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.926+0000 7f89ece00640 1 Processor -- start 2024-01-16T15:00:38.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.927+0000 7f89ece00640 1 -- start start 2024-01-16T15:00:38.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.928+0000 7f89ece00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89e806de10 0x7f89e815e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:38.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.928+0000 7f89ece00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f89e80c49f0 0x7f89e80705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:38.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.928+0000 7f89ece00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89e815ec90 con 0x7f89e8072160 2024-01-16T15:00:38.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.928+0000 7f89ece00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89e80cd530 con 0x7f89e80c49f0 2024-01-16T15:00:38.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.928+0000 7f89ece00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e80c4ed0 con 0x7f89e806de10 2024-01-16T15:00:38.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.929+0000 7f89dfe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89e806de10 0x7f89e815e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:38.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.929+0000 7f89dfe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89e806de10 0x7f89e815e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:36022/0 (socket says 172.21.15.143:36022) 2024-01-16T15:00:38.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.929+0000 7f89dfe00640 1 -- 172.21.15.143:0/572947482 learned_addr learned my addr 172.21.15.143:0/572947482 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:38.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.929+0000 7f89e6800640 1 --2- 172.21.15.143:0/572947482 >> v2:172.21.15.204:3300/0 conn(0x7f89e80c49f0 0x7f89e80705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:38.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.929+0000 7f89dfe00640 1 -- 172.21.15.143:0/572947482 >> v2:172.21.15.204:3300/0 conn(0x7f89e80c49f0 msgr2=0x7f89e80705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:38.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.929+0000 7f89dfe00640 1 --2- 172.21.15.143:0/572947482 >> v2:172.21.15.204:3300/0 conn(0x7f89e80c49f0 0x7f89e80705c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:38.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.930+0000 7f89dfe00640 1 -- 172.21.15.143:0/572947482 >> v1:172.21.15.204:6789/0 conn(0x7f89e8072160 legacy=0x7f89e806d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:38.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.930+0000 7f89dfe00640 1 -- 172.21.15.143:0/572947482 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e80c51b0 con 0x7f89e806de10 2024-01-16T15:00:38.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.931+0000 7f89dfe00640 1 --2- 172.21.15.143:0/572947482 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89e806de10 0x7f89e815e750 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f89d0009870 tx=0x7f89d002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82c3a877ebc62ab7 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:38.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.932+0000 7f89e5400640 1 -- 172.21.15.143:0/572947482 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d0030840 con 0x7f89e806de10 2024-01-16T15:00:38.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.932+0000 7f89e5400640 1 -- 172.21.15.143:0/572947482 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d00309a0 con 0x7f89e806de10 2024-01-16T15:00:38.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.932+0000 7f89e5400640 1 -- 172.21.15.143:0/572947482 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d0030d30 con 0x7f89e806de10 2024-01-16T15:00:38.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.933+0000 7f89ece00640 1 -- 172.21.15.143:0/572947482 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89e806de10 msgr2=0x7f89e815e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:38.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.933+0000 7f89ece00640 1 --2- 172.21.15.143:0/572947482 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89e806de10 0x7f89e815e750 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f89d0009870 tx=0x7f89d002b980 comp rx=0 tx=0).stop 2024-01-16T15:00:38.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.933+0000 7f89ece00640 1 -- 172.21.15.143:0/572947482 shutdown_connections 2024-01-16T15:00:38.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.933+0000 7f89ece00640 1 --2- 172.21.15.143:0/572947482 >> v2:172.21.15.204:3300/0 conn(0x7f89e80c49f0 0x7f89e80705c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:38.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.933+0000 7f89ece00640 1 --2- 172.21.15.143:0/572947482 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89e806de10 0x7f89e815e750 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:38.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.933+0000 7f89ece00640 1 -- 172.21.15.143:0/572947482 >> 172.21.15.143:0/572947482 conn(0x7f89e80b0c50 msgr2=0x7f89e80b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:38.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.933+0000 7f89ece00640 1 -- 172.21.15.143:0/572947482 shutdown_connections 2024-01-16T15:00:38.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.933+0000 7f89ece00640 1 -- 172.21.15.143:0/572947482 wait complete. 2024-01-16T15:00:38.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.934+0000 7f89ece00640 1 Processor -- start 2024-01-16T15:00:38.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.935+0000 7f89ece00640 1 -- start start 2024-01-16T15:00:38.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.935+0000 7f89ece00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89e806de10 0x7f89e8154070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:38.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.936+0000 7f89ece00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89e8072160 0x7f89e8159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:38.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.936+0000 7f89ece00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89e80c49f0 0x7f89e81f5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:38.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.936+0000 7f89ece00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e80c6540 con 0x7f89e806de10 2024-01-16T15:00:38.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.936+0000 7f89ece00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e80c6820 con 0x7f89e80c49f0 2024-01-16T15:00:38.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.936+0000 7f89ece00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e80c6b00 con 0x7f89e8072160 2024-01-16T15:00:38.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.936+0000 7f89e5e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89e806de10 0x7f89e8154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:38.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.936+0000 7f89e5e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89e806de10 0x7f89e8154070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43264/0 (socket says 172.21.15.143:43264) 2024-01-16T15:00:38.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.936+0000 7f89e5e00640 1 -- 172.21.15.143:0/518455092 learned_addr learned my addr 172.21.15.143:0/518455092 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:38.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.936+0000 7f89dfe00640 1 --2- 172.21.15.143:0/518455092 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89e8072160 0x7f89e8159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:38.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.936+0000 7f89e5e00640 1 -- 172.21.15.143:0/518455092 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89e8072160 msgr2=0x7f89e8159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:38.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.936+0000 7f89e6800640 1 --2- 172.21.15.143:0/518455092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89e80c49f0 0x7f89e81f5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:38.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.936+0000 7f89e5e00640 1 --2- 172.21.15.143:0/518455092 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89e8072160 0x7f89e8159900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:38.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.936+0000 7f89e5e00640 1 -- 172.21.15.143:0/518455092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89e80c49f0 msgr2=0x7f89e81f5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:38.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.936+0000 7f89e5e00640 1 --2- 172.21.15.143:0/518455092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89e80c49f0 0x7f89e81f5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:38.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.936+0000 7f89e5e00640 1 -- 172.21.15.143:0/518455092 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d00094d0 con 0x7f89e806de10 2024-01-16T15:00:38.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.937+0000 7f89e5e00640 1 --2- 172.21.15.143:0/518455092 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89e806de10 0x7f89e8154070 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f89d800c450 tx=0x7f89d800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:38.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.937+0000 7f89de000640 1 -- 172.21.15.143:0/518455092 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d80052d0 con 0x7f89e806de10 2024-01-16T15:00:38.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.937+0000 7f89de000640 1 -- 172.21.15.143:0/518455092 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d8005430 con 0x7f89e806de10 2024-01-16T15:00:38.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.937+0000 7f89de000640 1 -- 172.21.15.143:0/518455092 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d8012870 con 0x7f89e806de10 2024-01-16T15:00:38.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.937+0000 7f89ece00640 1 -- 172.21.15.143:0/518455092 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e80c6de0 con 0x7f89e806de10 2024-01-16T15:00:38.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.937+0000 7f89ece00640 1 -- 172.21.15.143:0/518455092 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e8159e40 con 0x7f89e806de10 2024-01-16T15:00:38.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.938+0000 7f89ece00640 1 -- 172.21.15.143:0/518455092 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89e8159fd0 con 0x7f89e806de10 2024-01-16T15:00:38.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.939+0000 7f89de000640 1 -- 172.21.15.143:0/518455092 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f89d80129d0 con 0x7f89e806de10 2024-01-16T15:00:38.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.940+0000 7f89de000640 1 --2- 172.21.15.143:0/518455092 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f89c006b2c0 0x7f89c006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:38.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.940+0000 7f89de000640 1 -- 172.21.15.143:0/518455092 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f89d8089630 con 0x7f89e806de10 2024-01-16T15:00:38.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.940+0000 7f89dfe00640 1 --2- 172.21.15.143:0/518455092 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f89c006b2c0 0x7f89c006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:38.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.941+0000 7f89dfe00640 1 --2- 172.21.15.143:0/518455092 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f89c006b2c0 0x7f89c006d780 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f89d0036000 tx=0x7f89d003b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:38.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:38.945+0000 7f89de000640 1 -- 172.21.15.143:0/518455092 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f89d8054400 con 0x7f89e806de10 2024-01-16T15:00:39.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.135+0000 7f89ece00640 1 -- 172.21.15.143:0/518455092 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f89e815ec90 con 0x7f89e806de10 2024-01-16T15:00:39.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.136+0000 7f89de000640 1 -- 172.21.15.143:0/518455092 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f89e815ec90 con 0x7f89e806de10 2024-01-16T15:00:39.116 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2024-01-16T15:00:39.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.138+0000 7f89ece00640 1 -- 172.21.15.143:0/518455092 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f89c006b2c0 msgr2=0x7f89c006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:39.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.138+0000 7f89ece00640 1 --2- 172.21.15.143:0/518455092 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f89c006b2c0 0x7f89c006d780 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f89d0036000 tx=0x7f89d003b040 comp rx=0 tx=0).stop 2024-01-16T15:00:39.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.138+0000 7f89ece00640 1 -- 172.21.15.143:0/518455092 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89e806de10 msgr2=0x7f89e8154070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:39.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.138+0000 7f89ece00640 1 --2- 172.21.15.143:0/518455092 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89e806de10 0x7f89e8154070 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f89d800c450 tx=0x7f89d800c910 comp rx=0 tx=0).stop 2024-01-16T15:00:39.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.138+0000 7f89ece00640 1 -- 172.21.15.143:0/518455092 shutdown_connections 2024-01-16T15:00:39.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.138+0000 7f89ece00640 1 --2- 172.21.15.143:0/518455092 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f89c006b2c0 0x7f89c006d780 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:39.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.138+0000 7f89ece00640 1 --2- 172.21.15.143:0/518455092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89e80c49f0 0x7f89e81f5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:39.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.138+0000 7f89ece00640 1 --2- 172.21.15.143:0/518455092 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89e8072160 0x7f89e8159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:39.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.138+0000 7f89ece00640 1 --2- 172.21.15.143:0/518455092 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89e806de10 0x7f89e8154070 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:39.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.138+0000 7f89ece00640 1 -- 172.21.15.143:0/518455092 >> 172.21.15.143:0/518455092 conn(0x7f89e80b0c50 msgr2=0x7f89e80b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:39.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.139+0000 7f89ece00640 1 -- 172.21.15.143:0/518455092 shutdown_connections 2024-01-16T15:00:39.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.139+0000 7f89ece00640 1 -- 172.21.15.143:0/518455092 wait complete. 2024-01-16T15:00:39.137 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-16T15:00:39.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.366+0000 7f2535800640 1 Processor -- start 2024-01-16T15:00:39.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.366+0000 7f2535800640 1 -- start start 2024-01-16T15:00:39.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.368+0000 7f2535800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2530140720 0x7f25300cb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:39.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.368+0000 7f2535800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2530161080 0x7f25300c60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:39.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.368+0000 7f2535800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2530161600 con 0x7f2530140340 2024-01-16T15:00:39.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.368+0000 7f2535800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25301618e0 con 0x7f2530161080 2024-01-16T15:00:39.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.368+0000 7f2535800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2530161bc0 con 0x7f2530140720 2024-01-16T15:00:39.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.368+0000 7f252f400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2530161080 0x7f25300c60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:39.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.368+0000 7f252f400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2530161080 0x7f25300c60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43268/0 (socket says 172.21.15.143:43268) 2024-01-16T15:00:39.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.368+0000 7f252f400640 1 -- 172.21.15.143:0/2458949973 learned_addr learned my addr 172.21.15.143:0/2458949973 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:39.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.369+0000 7f252e000640 1 --2- 172.21.15.143:0/2458949973 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2530140720 0x7f25300cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:39.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.369+0000 7f252f400640 1 -- 172.21.15.143:0/2458949973 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2530140720 msgr2=0x7f25300cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:39.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.369+0000 7f252f400640 1 --2- 172.21.15.143:0/2458949973 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2530140720 0x7f25300cb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:39.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.369+0000 7f252f400640 1 -- 172.21.15.143:0/2458949973 >> v1:172.21.15.204:6789/0 conn(0x7f2530140340 legacy=0x7f25300ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:39.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.369+0000 7f252f400640 1 -- 172.21.15.143:0/2458949973 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2530161ea0 con 0x7f2530161080 2024-01-16T15:00:39.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.369+0000 7f252e000640 1 --2- 172.21.15.143:0/2458949973 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2530140720 0x7f25300cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:39.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.370+0000 7f252f400640 1 --2- 172.21.15.143:0/2458949973 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2530161080 0x7f25300c60b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2528009870 tx=0x7f252802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47facfd126d955d8 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:39.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.370+0000 7f252d600640 1 -- 172.21.15.143:0/2458949973 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25280317c0 con 0x7f2530161080 2024-01-16T15:00:39.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.370+0000 7f252d600640 1 -- 172.21.15.143:0/2458949973 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2528031920 con 0x7f2530161080 2024-01-16T15:00:39.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.370+0000 7f252d600640 1 -- 172.21.15.143:0/2458949973 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2528031cb0 con 0x7f2530161080 2024-01-16T15:00:39.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.371+0000 7f2535800640 1 -- 172.21.15.143:0/2458949973 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2530161080 msgr2=0x7f25300c60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:39.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.371+0000 7f2535800640 1 --2- 172.21.15.143:0/2458949973 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2530161080 0x7f25300c60b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2528009870 tx=0x7f252802b860 comp rx=0 tx=0).stop 2024-01-16T15:00:39.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.371+0000 7f2535800640 1 -- 172.21.15.143:0/2458949973 shutdown_connections 2024-01-16T15:00:39.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.371+0000 7f2535800640 1 --2- 172.21.15.143:0/2458949973 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2530161080 0x7f25300c60b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:39.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.371+0000 7f2535800640 1 --2- 172.21.15.143:0/2458949973 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2530140720 0x7f25300cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:39.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.371+0000 7f2535800640 1 -- 172.21.15.143:0/2458949973 >> 172.21.15.143:0/2458949973 conn(0x7f25300b2e40 msgr2=0x7f25300b3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:39.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.371+0000 7f2535800640 1 -- 172.21.15.143:0/2458949973 shutdown_connections 2024-01-16T15:00:39.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.371+0000 7f2535800640 1 -- 172.21.15.143:0/2458949973 wait complete. 2024-01-16T15:00:39.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.372+0000 7f2535800640 1 Processor -- start 2024-01-16T15:00:39.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.373+0000 7f2535800640 1 -- start start 2024-01-16T15:00:39.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.373+0000 7f2535800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2530140340 0x7f2530160670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:39.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.374+0000 7f2535800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2530140720 0x7f25300b6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:39.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.374+0000 7f2535800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2530161080 0x7f25301f6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:39.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.374+0000 7f2535800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25300c8a40 con 0x7f2530140340 2024-01-16T15:00:39.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.374+0000 7f2535800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25300b7250 con 0x7f2530140720 2024-01-16T15:00:39.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.374+0000 7f2535800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25300b6190 con 0x7f2530161080 2024-01-16T15:00:39.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.374+0000 7f252ea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2530140340 0x7f2530160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:39.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.374+0000 7f252ea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2530140340 0x7f2530160670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43274/0 (socket says 172.21.15.143:43274) 2024-01-16T15:00:39.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.374+0000 7f252ea00640 1 -- 172.21.15.143:0/1410319617 learned_addr learned my addr 172.21.15.143:0/1410319617 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:39.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.374+0000 7f252e000640 1 --2- 172.21.15.143:0/1410319617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2530140720 0x7f25300b6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:39.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.374+0000 7f252ea00640 1 -- 172.21.15.143:0/1410319617 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2530161080 msgr2=0x7f25301f6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:39.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.374+0000 7f252ea00640 1 --2- 172.21.15.143:0/1410319617 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2530161080 0x7f25301f6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:39.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.374+0000 7f252ea00640 1 -- 172.21.15.143:0/1410319617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2530140720 msgr2=0x7f25300b6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:39.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.374+0000 7f252ea00640 1 --2- 172.21.15.143:0/1410319617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2530140720 0x7f25300b6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:39.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.374+0000 7f252ea00640 1 -- 172.21.15.143:0/1410319617 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25280094d0 con 0x7f2530140340 2024-01-16T15:00:39.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.375+0000 7f252ea00640 1 --2- 172.21.15.143:0/1410319617 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2530140340 0x7f2530160670 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f251800c430 tx=0x7f251800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:39.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.375+0000 7f2526a00640 1 -- 172.21.15.143:0/1410319617 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2518011790 con 0x7f2530140340 2024-01-16T15:00:39.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.375+0000 7f2526a00640 1 -- 172.21.15.143:0/1410319617 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25180118f0 con 0x7f2530140340 2024-01-16T15:00:39.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.375+0000 7f2526a00640 1 -- 172.21.15.143:0/1410319617 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2518011ce0 con 0x7f2530140340 2024-01-16T15:00:39.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.375+0000 7f2535800640 1 -- 172.21.15.143:0/1410319617 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25300b6470 con 0x7f2530140340 2024-01-16T15:00:39.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.375+0000 7f2535800640 1 -- 172.21.15.143:0/1410319617 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25300c8660 con 0x7f2530140340 2024-01-16T15:00:39.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.376+0000 7f2535800640 1 -- 172.21.15.143:0/1410319617 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2530067990 con 0x7f2530140340 2024-01-16T15:00:39.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.377+0000 7f2526a00640 1 -- 172.21.15.143:0/1410319617 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f251801c460 con 0x7f2530140340 2024-01-16T15:00:39.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.378+0000 7f2526a00640 1 --2- 172.21.15.143:0/1410319617 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f250806b2c0 0x7f250806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:39.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.378+0000 7f2526a00640 1 -- 172.21.15.143:0/1410319617 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2518089b00 con 0x7f2530140340 2024-01-16T15:00:39.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.378+0000 7f252e000640 1 --2- 172.21.15.143:0/1410319617 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f250806b2c0 0x7f250806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:39.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.379+0000 7f252e000640 1 --2- 172.21.15.143:0/1410319617 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f250806b2c0 0x7f250806d780 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f25300c71b0 tx=0x7f25200084d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:39.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.383+0000 7f2526a00640 1 -- 172.21.15.143:0/1410319617 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2518054a00 con 0x7f2530140340 2024-01-16T15:00:39.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.574+0000 7f2535800640 1 -- 172.21.15.143:0/1410319617 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f253004e8b0 con 0x7f2530140340 2024-01-16T15:00:39.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.575+0000 7f2526a00640 1 -- 172.21.15.143:0/1410319617 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2518058020 con 0x7f2530140340 2024-01-16T15:00:39.555 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2024-01-16T15:00:39.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.577+0000 7f2535800640 1 -- 172.21.15.143:0/1410319617 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f250806b2c0 msgr2=0x7f250806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:39.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.577+0000 7f2535800640 1 --2- 172.21.15.143:0/1410319617 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f250806b2c0 0x7f250806d780 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f25300c71b0 tx=0x7f25200084d0 comp rx=0 tx=0).stop 2024-01-16T15:00:39.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.577+0000 7f2535800640 1 -- 172.21.15.143:0/1410319617 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2530140340 msgr2=0x7f2530160670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:39.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.577+0000 7f2535800640 1 --2- 172.21.15.143:0/1410319617 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2530140340 0x7f2530160670 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f251800c430 tx=0x7f251800c8f0 comp rx=0 tx=0).stop 2024-01-16T15:00:39.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.578+0000 7f2535800640 1 -- 172.21.15.143:0/1410319617 shutdown_connections 2024-01-16T15:00:39.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.578+0000 7f2535800640 1 --2- 172.21.15.143:0/1410319617 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f250806b2c0 0x7f250806d780 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:39.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.578+0000 7f2535800640 1 --2- 172.21.15.143:0/1410319617 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2530161080 0x7f25301f6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:39.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.578+0000 7f2535800640 1 --2- 172.21.15.143:0/1410319617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2530140720 0x7f25300b6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:39.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.578+0000 7f2535800640 1 --2- 172.21.15.143:0/1410319617 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2530140340 0x7f2530160670 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:39.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.578+0000 7f2535800640 1 -- 172.21.15.143:0/1410319617 >> 172.21.15.143:0/1410319617 conn(0x7f25300b2e40 msgr2=0x7f253006caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:39.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.578+0000 7f2535800640 1 -- 172.21.15.143:0/1410319617 shutdown_connections 2024-01-16T15:00:39.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.578+0000 7f2535800640 1 -- 172.21.15.143:0/1410319617 wait complete. 2024-01-16T15:00:39.575 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T15:00:39.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.797+0000 7f2ad6c00640 1 Processor -- start 2024-01-16T15:00:39.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.797+0000 7f2ad6c00640 1 -- start start 2024-01-16T15:00:39.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.799+0000 7f2ad6c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2ad015c7f0 0x7f2ad006db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:39.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.799+0000 7f2ad6c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2ad00ce6c0 0x7f2ad00cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:39.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.799+0000 7f2ad6c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ad00cece0 con 0x7f2ad00cf450 2024-01-16T15:00:39.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.799+0000 7f2ad6c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ad00cfe40 con 0x7f2ad00ce6c0 2024-01-16T15:00:39.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.799+0000 7f2ad6c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ad00ceaf0 con 0x7f2ad015c7f0 2024-01-16T15:00:39.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.799+0000 7f2ad4c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2ad00ce6c0 0x7f2ad00cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:39.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.799+0000 7f2ad4c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2ad00ce6c0 0x7f2ad00cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43288/0 (socket says 172.21.15.143:43288) 2024-01-16T15:00:39.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.799+0000 7f2ad4c00640 1 -- 172.21.15.143:0/1985178208 learned_addr learned my addr 172.21.15.143:0/1985178208 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:39.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.799+0000 7f2acf400640 1 --1- 172.21.15.143:0/1985178208 >> v1:172.21.15.143:6789/0 conn(0x7f2ad00cf450 0x7f2ad00cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49258/0 (socket says 172.21.15.143:49258) 2024-01-16T15:00:39.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.799+0000 7f2acfe00640 1 --2- 172.21.15.143:0/1985178208 >> v2:172.21.15.204:3300/0 conn(0x7f2ad015c7f0 0x7f2ad006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:39.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.800+0000 7f2ad4c00640 1 -- 172.21.15.143:0/1985178208 >> v2:172.21.15.204:3300/0 conn(0x7f2ad015c7f0 msgr2=0x7f2ad006db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:39.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.800+0000 7f2ad4c00640 1 --2- 172.21.15.143:0/1985178208 >> v2:172.21.15.204:3300/0 conn(0x7f2ad015c7f0 0x7f2ad006db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:39.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.800+0000 7f2ad4c00640 1 -- 172.21.15.143:0/1985178208 >> v1:172.21.15.143:6789/0 conn(0x7f2ad00cf450 legacy=0x7f2ad00cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:39.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.800+0000 7f2ad4c00640 1 -- 172.21.15.143:0/1985178208 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad006d040 con 0x7f2ad00ce6c0 2024-01-16T15:00:39.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.801+0000 7f2ad4c00640 1 --2- 172.21.15.143:0/1985178208 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ad00ce6c0 0x7f2ad00cf900 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2ad00d0130 tx=0x7f2ac802bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de79ea1432099c1f server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:39.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.801+0000 7f2acea00640 1 -- 172.21.15.143:0/1985178208 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac802cb10 con 0x7f2ad00ce6c0 2024-01-16T15:00:39.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.801+0000 7f2acea00640 1 -- 172.21.15.143:0/1985178208 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ac802cc70 con 0x7f2ad00ce6c0 2024-01-16T15:00:39.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.801+0000 7f2acea00640 1 -- 172.21.15.143:0/1985178208 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac80343f0 con 0x7f2ad00ce6c0 2024-01-16T15:00:39.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.802+0000 7f2ad6c00640 1 -- 172.21.15.143:0/1985178208 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ad00ce6c0 msgr2=0x7f2ad00cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:39.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.802+0000 7f2ad6c00640 1 --2- 172.21.15.143:0/1985178208 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ad00ce6c0 0x7f2ad00cf900 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2ad00d0130 tx=0x7f2ac802bf70 comp rx=0 tx=0).stop 2024-01-16T15:00:39.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.802+0000 7f2ad6c00640 1 -- 172.21.15.143:0/1985178208 shutdown_connections 2024-01-16T15:00:39.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.802+0000 7f2ad6c00640 1 --2- 172.21.15.143:0/1985178208 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ad00ce6c0 0x7f2ad00cf900 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:39.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.802+0000 7f2ad6c00640 1 --2- 172.21.15.143:0/1985178208 >> v2:172.21.15.204:3300/0 conn(0x7f2ad015c7f0 0x7f2ad006db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:39.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.802+0000 7f2ad6c00640 1 -- 172.21.15.143:0/1985178208 >> 172.21.15.143:0/1985178208 conn(0x7f2ad006a920 msgr2=0x7f2ad006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:39.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.802+0000 7f2ad6c00640 1 -- 172.21.15.143:0/1985178208 shutdown_connections 2024-01-16T15:00:39.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.802+0000 7f2ad6c00640 1 -- 172.21.15.143:0/1985178208 wait complete. 2024-01-16T15:00:39.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.804+0000 7f2ad6c00640 1 Processor -- start 2024-01-16T15:00:39.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.804+0000 7f2ad6c00640 1 -- start start 2024-01-16T15:00:39.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.805+0000 7f2ad6c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ad00ce6c0 0x7f2ad014e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:39.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.805+0000 7f2ad6c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ad00cf450 0x7f2ad0155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:39.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.805+0000 7f2ad6c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ad015c7f0 0x7f2ad00abfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:39.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.805+0000 7f2ad6c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad014ee20 con 0x7f2ad015c7f0 2024-01-16T15:00:39.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.805+0000 7f2ad6c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad0155b50 con 0x7f2ad00ce6c0 2024-01-16T15:00:39.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.805+0000 7f2ad6c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad0154a00 con 0x7f2ad00cf450 2024-01-16T15:00:39.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.805+0000 7f2ad4c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ad015c7f0 0x7f2ad00abfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:39.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.805+0000 7f2ad4c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ad015c7f0 0x7f2ad00abfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43298/0 (socket says 172.21.15.143:43298) 2024-01-16T15:00:39.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.805+0000 7f2ad4c00640 1 -- 172.21.15.143:0/2551767484 learned_addr learned my addr 172.21.15.143:0/2551767484 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:39.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.805+0000 7f2acfe00640 1 --2- 172.21.15.143:0/2551767484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ad00ce6c0 0x7f2ad014e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:39.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.805+0000 7f2ad4c00640 1 -- 172.21.15.143:0/2551767484 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ad00cf450 msgr2=0x7f2ad0155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:39.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.805+0000 7f2ad4c00640 1 --2- 172.21.15.143:0/2551767484 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ad00cf450 0x7f2ad0155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:39.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.806+0000 7f2ad4c00640 1 -- 172.21.15.143:0/2551767484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ad00ce6c0 msgr2=0x7f2ad014e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:39.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.806+0000 7f2ad4c00640 1 --2- 172.21.15.143:0/2551767484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ad00ce6c0 0x7f2ad014e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:39.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.806+0000 7f2ad4c00640 1 -- 172.21.15.143:0/2551767484 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac8009b30 con 0x7f2ad015c7f0 2024-01-16T15:00:39.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.806+0000 7f2ad4c00640 1 --2- 172.21.15.143:0/2551767484 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ad015c7f0 0x7f2ad00abfc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8036040 tx=0x7f2ac8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:39.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.806+0000 7f2ac7e00640 1 -- 172.21.15.143:0/2551767484 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac8009510 con 0x7f2ad015c7f0 2024-01-16T15:00:39.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.806+0000 7f2ac7e00640 1 -- 172.21.15.143:0/2551767484 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ac803e450 con 0x7f2ad015c7f0 2024-01-16T15:00:39.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.806+0000 7f2ad6c00640 1 -- 172.21.15.143:0/2551767484 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ad014f4f0 con 0x7f2ad015c7f0 2024-01-16T15:00:39.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.806+0000 7f2ac7e00640 1 -- 172.21.15.143:0/2551767484 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac803e800 con 0x7f2ad015c7f0 2024-01-16T15:00:39.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.806+0000 7f2ad6c00640 1 -- 172.21.15.143:0/2551767484 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ad014fab0 con 0x7f2ad015c7f0 2024-01-16T15:00:39.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.808+0000 7f2ad6c00640 1 -- 172.21.15.143:0/2551767484 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ad0067470 con 0x7f2ad015c7f0 2024-01-16T15:00:39.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.808+0000 7f2ac7e00640 1 -- 172.21.15.143:0/2551767484 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f2ac803e960 con 0x7f2ad015c7f0 2024-01-16T15:00:39.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.809+0000 7f2ac7e00640 1 --2- 172.21.15.143:0/2551767484 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f2aa806b2c0 0x7f2aa806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:39.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.809+0000 7f2ac7e00640 1 -- 172.21.15.143:0/2551767484 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2ac80ac940 con 0x7f2ad015c7f0 2024-01-16T15:00:39.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.809+0000 7f2acfe00640 1 --2- 172.21.15.143:0/2551767484 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f2aa806b2c0 0x7f2aa806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:39.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.810+0000 7f2acfe00640 1 --2- 172.21.15.143:0/2551767484 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f2aa806b2c0 0x7f2aa806d780 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8006fd0 tx=0x7f2ab8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:39.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:39.813+0000 7f2ac7e00640 1 -- 172.21.15.143:0/2551767484 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2ac80777a0 con 0x7f2ad015c7f0 2024-01-16T15:00:39.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.004+0000 7f2ad6c00640 1 -- 172.21.15.143:0/2551767484 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ad00cd730 con 0x7f2ad015c7f0 2024-01-16T15:00:39.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.007+0000 7f2ac7e00640 1 -- 172.21.15.143:0/2551767484 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16565 (secure 0 0 0) 0x7f2ac807adc0 con 0x7f2ad015c7f0 2024-01-16T15:00:39.987 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:00:39.987 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0","created":"2024-01-16T14:59:38.532601+0000","modified":"2024-01-16T15:00:04.543620+0000","last_up_change":"2024-01-16T14:59:50.983699+0000","last_in_change":"2024-01-16T14:59:42.056297+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:59:51.727720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T15:00:01.010472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T15:00:02.041228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f4c8891d-ab29-472a-907d-8446f20f7012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6801","nonce":1569293626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6805","nonce":1569293626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6814","nonce":1569293626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6809","nonce":1569293626}]},"public_addr":"172.21.15.143:6801/1569293626","cluster_addr":"172.21.15.143:6805/1569293626","heartbeat_back_addr":"172.21.15.143:6814/1569293626","heartbeat_front_addr":"172.21.15.143:6809/1569293626","state":["exists","up"]},{"osd":1,"uuid":"c4a2788d-8b89-4992-82b2-7f4b9debd509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6804","nonce":1276841168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6808","nonce":1276841168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6813","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6815","nonce":1276841168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6811","nonce":1276841168}]},"public_addr":"172.21.15.143:6804/1276841168","cluster_addr":"172.21.15.143:6808/1276841168","heartbeat_back_addr":"172.21.15.143:6815/1276841168","heartbeat_front_addr":"172.21.15.143:6811/1276841168","state":["exists","up"]},{"osd":2,"uuid":"64645c1d-31b2-4794-b0f2-7ccf8df30393","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6817","nonce":1217670170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6819","nonce":1217670170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6823","nonce":1217670170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6821","nonce":1217670170}]},"public_addr":"172.21.15.143:6817/1217670170","cluster_addr":"172.21.15.143:6819/1217670170","heartbeat_back_addr":"172.21.15.143:6823/1217670170","heartbeat_front_addr":"172.21.15.143:6821/1217670170","state":["exists","up"]},{"osd":3,"uuid":"73674129-a84e-4d31-8247-7d0179ae6df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6825","nonce":3414188012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6827","nonce":3414188012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6831","nonce":3414188012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6829","nonce":3414188012}]},"public_addr":"172.21.15.143:6825/3414188012","cluster_addr":"172.21.15.143:6827/3414188012","heartbeat_back_addr":"172.21.15.143:6831/3414188012","heartbeat_front_addr":"172.21.15.143:6829/3414188012","state":["exists","up"]},{"osd":4,"uuid":"cea28c9b-2f1c-4087-b415-9960789131d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6811","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6812","nonce":1070255716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6816","nonce":1070255716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6823","nonce":1070255716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6820","nonce":1070255716}]},"public_addr":"172.21.15.204:6812/1070255716","cluster_addr":"172.21.15.204:6816/1070255716","heartbeat_back_addr":"172.21.15.204:6823/1070255716","heartbeat_front_addr":"172.21.15.204:6820/1070255716","state":["exists","up"]},{"osd":5,"uuid":"33b63560-5574-4e78-983f-b398d67a2cb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6809","nonce":3605417837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6813","nonce":3605417837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6819","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6821","nonce":3605417837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6817","nonce":3605417837}]},"public_addr":"172.21.15.204:6809/3605417837","cluster_addr":"172.21.15.204:6813/3605417837","heartbeat_back_addr":"172.21.15.204:6821/3605417837","heartbeat_front_addr":"172.21.15.204:6817/3605417837","state":["exists","up"]},{"osd":6,"uuid":"df397427-8940-4eee-9c79-eece669d735d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6801","nonce":1085000534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6803","nonce":1085000534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6808","nonce":1085000534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6805","nonce":1085000534}]},"public_addr":"172.21.15.204:6801/1085000534","cluster_addr":"172.21.15.204:6803/1085000534","heartbeat_back_addr":"172.21.15.204:6808/1085000534","heartbeat_front_addr":"172.21.15.204:6805/1085000534","state":["exists","up"]},{"osd":7,"uuid":"a2e5a15c-8312-405a-ac02-f7e1fc9df2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6825","nonce":1337220609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6827","nonce":1337220609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6831","nonce":1337220609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6829","nonce":1337220609}]},"public_addr":"172.21.15.204:6825/1337220609","cluster_addr":"172.21.15.204:6827/1337220609","heartbeat_back_addr":"172.21.15.204:6831/1337220609","heartbeat_front_addr":"172.21.15.204:6829/1337220609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:49.984507+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.395306+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.632998+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.373627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.342920+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.438366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.331926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.406657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T15:00:39.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.009+0000 7f2ad6c00640 1 -- 172.21.15.143:0/2551767484 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f2aa806b2c0 msgr2=0x7f2aa806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:39.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.009+0000 7f2ad6c00640 1 --2- 172.21.15.143:0/2551767484 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f2aa806b2c0 0x7f2aa806d780 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8006fd0 tx=0x7f2ab8008040 comp rx=0 tx=0).stop 2024-01-16T15:00:39.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.009+0000 7f2ad6c00640 1 -- 172.21.15.143:0/2551767484 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ad015c7f0 msgr2=0x7f2ad00abfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:39.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.009+0000 7f2ad6c00640 1 --2- 172.21.15.143:0/2551767484 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ad015c7f0 0x7f2ad00abfc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8036040 tx=0x7f2ac8004a40 comp rx=0 tx=0).stop 2024-01-16T15:00:39.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.010+0000 7f2ad6c00640 1 -- 172.21.15.143:0/2551767484 shutdown_connections 2024-01-16T15:00:39.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.010+0000 7f2ad6c00640 1 --2- 172.21.15.143:0/2551767484 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f2aa806b2c0 0x7f2aa806d780 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:39.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.010+0000 7f2ad6c00640 1 --2- 172.21.15.143:0/2551767484 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ad015c7f0 0x7f2ad00abfc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:39.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.010+0000 7f2ad6c00640 1 --2- 172.21.15.143:0/2551767484 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ad00cf450 0x7f2ad0155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:39.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.010+0000 7f2ad6c00640 1 --2- 172.21.15.143:0/2551767484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ad00ce6c0 0x7f2ad014e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:39.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.010+0000 7f2ad6c00640 1 -- 172.21.15.143:0/2551767484 >> 172.21.15.143:0/2551767484 conn(0x7f2ad006a920 msgr2=0x7f2ad015eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:39.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.010+0000 7f2ad6c00640 1 -- 172.21.15.143:0/2551767484 shutdown_connections 2024-01-16T15:00:39.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.010+0000 7f2ad6c00640 1 -- 172.21.15.143:0/2551767484 wait complete. 2024-01-16T15:00:40.008 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-16T14:59:51.727720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-16T15:00:01.010472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-16T15:00:02.041228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-16T15:00:40.009 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-16T15:00:40.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.236+0000 7fe468000640 1 Processor -- start 2024-01-16T15:00:40.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.236+0000 7fe468000640 1 -- start start 2024-01-16T15:00:40.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.237+0000 7fe468000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe4600b3330 0x7fe46006a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:40.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.237+0000 7fe468000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe4600b3790 0x7fe46006abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:40.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.237+0000 7fe468000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe46006dcf0 con 0x7fe460161060 2024-01-16T15:00:40.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.237+0000 7fe468000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe46006ec10 con 0x7fe4600b3330 2024-01-16T15:00:40.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.237+0000 7fe468000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe460161ec0 con 0x7fe4600b3790 2024-01-16T15:00:40.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.238+0000 7fe466000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe4600b3790 0x7fe46006abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:40.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.238+0000 7fe466000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe4600b3790 0x7fe46006abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:36078/0 (socket says 172.21.15.143:36078) 2024-01-16T15:00:40.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.238+0000 7fe464c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe4600b3330 0x7fe46006a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:40.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.238+0000 7fe466000640 1 -- 172.21.15.143:0/2149883775 learned_addr learned my addr 172.21.15.143:0/2149883775 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:40.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.238+0000 7fe465600640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fe460161060 0x7fe460161440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:39258/0 (socket says 172.21.15.143:39258) 2024-01-16T15:00:40.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.239+0000 7fe464c00640 1 -- 172.21.15.143:0/2149883775 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe4600b3790 msgr2=0x7fe46006abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:40.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.239+0000 7fe464c00640 1 --2- 172.21.15.143:0/2149883775 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe4600b3790 0x7fe46006abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:40.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.239+0000 7fe464c00640 1 -- 172.21.15.143:0/2149883775 >> v1:172.21.15.204:6789/0 conn(0x7fe460161060 legacy=0x7fe460161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:40.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.239+0000 7fe464c00640 1 -- 172.21.15.143:0/2149883775 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe46006df90 con 0x7fe4600b3330 2024-01-16T15:00:40.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.239+0000 7fe466000640 1 --2- 172.21.15.143:0/2149883775 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe4600b3790 0x7fe46006abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T15:00:40.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.240+0000 7fe464c00640 1 --2- 172.21.15.143:0/2149883775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4600b3330 0x7fe46006a6b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe448009790 tx=0x7fe44802b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad44ca51c96ceefa server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:40.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.241+0000 7fe45fe00640 1 -- 172.21.15.143:0/2149883775 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe448031770 con 0x7fe4600b3330 2024-01-16T15:00:40.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.241+0000 7fe45fe00640 1 -- 172.21.15.143:0/2149883775 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4480318d0 con 0x7fe4600b3330 2024-01-16T15:00:40.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.241+0000 7fe45fe00640 1 -- 172.21.15.143:0/2149883775 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe448031c60 con 0x7fe4600b3330 2024-01-16T15:00:40.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.241+0000 7fe468000640 1 -- 172.21.15.143:0/2149883775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4600b3330 msgr2=0x7fe46006a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:40.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.241+0000 7fe468000640 1 --2- 172.21.15.143:0/2149883775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4600b3330 0x7fe46006a6b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe448009790 tx=0x7fe44802b810 comp rx=0 tx=0).stop 2024-01-16T15:00:40.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.241+0000 7fe468000640 1 -- 172.21.15.143:0/2149883775 shutdown_connections 2024-01-16T15:00:40.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.241+0000 7fe468000640 1 --2- 172.21.15.143:0/2149883775 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe4600b3790 0x7fe46006abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:40.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.241+0000 7fe468000640 1 --2- 172.21.15.143:0/2149883775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4600b3330 0x7fe46006a6b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:40.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.241+0000 7fe468000640 1 -- 172.21.15.143:0/2149883775 >> 172.21.15.143:0/2149883775 conn(0x7fe4600cf900 msgr2=0x7fe4600cfd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:40.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.242+0000 7fe468000640 1 -- 172.21.15.143:0/2149883775 shutdown_connections 2024-01-16T15:00:40.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.242+0000 7fe468000640 1 -- 172.21.15.143:0/2149883775 wait complete. 2024-01-16T15:00:40.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.244+0000 7fe468000640 1 Processor -- start 2024-01-16T15:00:40.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.244+0000 7fe468000640 1 -- start start 2024-01-16T15:00:40.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.245+0000 7fe468000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4600b3330 0x7fe460144f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:40.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.245+0000 7fe468000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe4600b3790 0x7fe460142e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:40.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.245+0000 7fe468000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe460161060 0x7fe46015dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:40.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.245+0000 7fe468000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe460144120 con 0x7fe460161060 2024-01-16T15:00:40.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.245+0000 7fe468000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe460143510 con 0x7fe4600b3330 2024-01-16T15:00:40.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.245+0000 7fe468000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4601437f0 con 0x7fe4600b3790 2024-01-16T15:00:40.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.245+0000 7fe466000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe460161060 0x7fe46015dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:40.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.245+0000 7fe466000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe460161060 0x7fe46015dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43300/0 (socket says 172.21.15.143:43300) 2024-01-16T15:00:40.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.245+0000 7fe466000640 1 -- 172.21.15.143:0/1070437106 learned_addr learned my addr 172.21.15.143:0/1070437106 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:40.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.246+0000 7fe464c00640 1 --2- 172.21.15.143:0/1070437106 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe4600b3790 0x7fe460142e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:40.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.246+0000 7fe465600640 1 --2- 172.21.15.143:0/1070437106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4600b3330 0x7fe460144f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:40.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.246+0000 7fe466000640 1 -- 172.21.15.143:0/1070437106 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe4600b3790 msgr2=0x7fe460142e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:40.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.246+0000 7fe466000640 1 --2- 172.21.15.143:0/1070437106 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe4600b3790 0x7fe460142e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:40.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.246+0000 7fe466000640 1 -- 172.21.15.143:0/1070437106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4600b3330 msgr2=0x7fe460144f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:40.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.246+0000 7fe466000640 1 --2- 172.21.15.143:0/1070437106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4600b3330 0x7fe460144f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:40.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.246+0000 7fe466000640 1 -- 172.21.15.143:0/1070437106 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4580094d0 con 0x7fe460161060 2024-01-16T15:00:40.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.246+0000 7fe466000640 1 --2- 172.21.15.143:0/1070437106 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe460161060 0x7fe46015dac0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe45800e890 tx=0x7fe45800ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:40.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.246+0000 7fe45d600640 1 -- 172.21.15.143:0/1070437106 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe458004020 con 0x7fe460161060 2024-01-16T15:00:40.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.247+0000 7fe45d600640 1 -- 172.21.15.143:0/1070437106 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe458004180 con 0x7fe460161060 2024-01-16T15:00:40.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.247+0000 7fe468000640 1 -- 172.21.15.143:0/1070437106 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4480093a0 con 0x7fe460161060 2024-01-16T15:00:40.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.247+0000 7fe45d600640 1 -- 172.21.15.143:0/1070437106 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45800c940 con 0x7fe460161060 2024-01-16T15:00:40.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.247+0000 7fe468000640 1 -- 172.21.15.143:0/1070437106 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe460143db0 con 0x7fe460161060 2024-01-16T15:00:40.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.248+0000 7fe468000640 1 -- 172.21.15.143:0/1070437106 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe46004e8b0 con 0x7fe460161060 2024-01-16T15:00:40.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.249+0000 7fe45d600640 1 -- 172.21.15.143:0/1070437106 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fe45800caa0 con 0x7fe460161060 2024-01-16T15:00:40.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.250+0000 7fe45d600640 1 --2- 172.21.15.143:0/1070437106 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe43806b2c0 0x7fe43806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:40.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.250+0000 7fe45d600640 1 -- 172.21.15.143:0/1070437106 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe458089ae0 con 0x7fe460161060 2024-01-16T15:00:40.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.250+0000 7fe465600640 1 --2- 172.21.15.143:0/1070437106 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe43806b2c0 0x7fe43806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:40.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.251+0000 7fe465600640 1 --2- 172.21.15.143:0/1070437106 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe43806b2c0 0x7fe43806d780 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe4500012e0 tx=0x7fe450005b90 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:40.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.254+0000 7fe45d600640 1 -- 172.21.15.143:0/1070437106 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe4580548b0 con 0x7fe460161060 2024-01-16T15:00:40.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.443+0000 7fe468000640 1 -- 172.21.15.143:0/1070437106 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe46006d0c0 con 0x7fe460161060 2024-01-16T15:00:40.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.444+0000 7fe45d600640 1 -- 172.21.15.143:0/1070437106 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe458057ed0 con 0x7fe460161060 2024-01-16T15:00:40.424 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-16T15:00:40.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.447+0000 7fe468000640 1 -- 172.21.15.143:0/1070437106 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe43806b2c0 msgr2=0x7fe43806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:40.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.447+0000 7fe468000640 1 --2- 172.21.15.143:0/1070437106 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe43806b2c0 0x7fe43806d780 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe4500012e0 tx=0x7fe450005b90 comp rx=0 tx=0).stop 2024-01-16T15:00:40.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.447+0000 7fe468000640 1 -- 172.21.15.143:0/1070437106 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe460161060 msgr2=0x7fe46015dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:40.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.447+0000 7fe468000640 1 --2- 172.21.15.143:0/1070437106 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe460161060 0x7fe46015dac0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe45800e890 tx=0x7fe45800ed50 comp rx=0 tx=0).stop 2024-01-16T15:00:40.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.447+0000 7fe468000640 1 -- 172.21.15.143:0/1070437106 shutdown_connections 2024-01-16T15:00:40.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.447+0000 7fe468000640 1 --2- 172.21.15.143:0/1070437106 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe43806b2c0 0x7fe43806d780 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:40.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.447+0000 7fe468000640 1 --2- 172.21.15.143:0/1070437106 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe460161060 0x7fe46015dac0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:40.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.447+0000 7fe468000640 1 --2- 172.21.15.143:0/1070437106 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe4600b3790 0x7fe460142e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:40.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.447+0000 7fe468000640 1 --2- 172.21.15.143:0/1070437106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4600b3330 0x7fe460144f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:40.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.447+0000 7fe468000640 1 -- 172.21.15.143:0/1070437106 >> 172.21.15.143:0/1070437106 conn(0x7fe4600cf900 msgr2=0x7fe46015eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:40.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.447+0000 7fe468000640 1 -- 172.21.15.143:0/1070437106 shutdown_connections 2024-01-16T15:00:40.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.447+0000 7fe468000640 1 -- 172.21.15.143:0/1070437106 wait complete. 2024-01-16T15:00:40.448 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-16T15:00:40.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.677+0000 7ff34e400640 1 Processor -- start 2024-01-16T15:00:40.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.677+0000 7ff34e400640 1 -- start start 2024-01-16T15:00:40.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.678+0000 7ff34e400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff34815b260 0x7ff34806e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:40.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.678+0000 7ff34e400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff3480b1250 0x7ff34806c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:40.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.679+0000 7ff34e400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff34815ed20 con 0x7ff34806e580 2024-01-16T15:00:40.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.679+0000 7ff34e400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff34815b640 con 0x7ff3480b1250 2024-01-16T15:00:40.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.679+0000 7ff34e400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff34815f8a0 con 0x7ff34815b260 2024-01-16T15:00:40.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.679+0000 7ff34da00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff3480b1250 0x7ff34806c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:40.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.679+0000 7ff347e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff34806e580 0x7ff3480b0b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49264/0 (socket says 172.21.15.143:49264) 2024-01-16T15:00:40.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.679+0000 7ff347e00640 1 -- 172.21.15.143:0/53324061 learned_addr learned my addr 172.21.15.143:0/53324061 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:40.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.679+0000 7ff34d000640 1 --2- 172.21.15.143:0/53324061 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff34815b260 0x7ff34806e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:40.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.679+0000 7ff347400640 1 -- 172.21.15.143:0/53324061 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3494197953 0 0) 0x7ff34815ed20 con 0x7ff34806e580 2024-01-16T15:00:40.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.679+0000 7ff347400640 1 -- 172.21.15.143:0/53324061 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff328003520 con 0x7ff34806e580 2024-01-16T15:00:40.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.679+0000 7ff34da00640 1 -- 172.21.15.143:0/53324061 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff34815b260 msgr2=0x7ff34806e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:40.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.679+0000 7ff34da00640 1 --2- 172.21.15.143:0/53324061 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff34815b260 0x7ff34806e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:40.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.679+0000 7ff34da00640 1 -- 172.21.15.143:0/53324061 >> v1:172.21.15.143:6789/0 conn(0x7ff34806e580 legacy=0x7ff3480b0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:40.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.679+0000 7ff34da00640 1 -- 172.21.15.143:0/53324061 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3480abe50 con 0x7ff3480b1250 2024-01-16T15:00:40.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.681+0000 7ff34da00640 1 --2- 172.21.15.143:0/53324061 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3480b1250 0x7ff34806c5b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff340009d90 tx=0x7ff34002bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e8cc20b5c1a5bb server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:40.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.681+0000 7ff347400640 1 -- 172.21.15.143:0/53324061 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34002cb30 con 0x7ff3480b1250 2024-01-16T15:00:40.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.681+0000 7ff347400640 1 -- 172.21.15.143:0/53324061 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff34002cc90 con 0x7ff3480b1250 2024-01-16T15:00:40.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.681+0000 7ff347400640 1 -- 172.21.15.143:0/53324061 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff340034440 con 0x7ff3480b1250 2024-01-16T15:00:40.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.681+0000 7ff34e400640 1 -- 172.21.15.143:0/53324061 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3480b1250 msgr2=0x7ff34806c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:40.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.681+0000 7ff34e400640 1 --2- 172.21.15.143:0/53324061 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3480b1250 0x7ff34806c5b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff340009d90 tx=0x7ff34002bd80 comp rx=0 tx=0).stop 2024-01-16T15:00:40.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.681+0000 7ff34e400640 1 -- 172.21.15.143:0/53324061 shutdown_connections 2024-01-16T15:00:40.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.681+0000 7ff34e400640 1 --2- 172.21.15.143:0/53324061 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3480b1250 0x7ff34806c5b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:40.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.681+0000 7ff34e400640 1 --2- 172.21.15.143:0/53324061 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff34815b260 0x7ff34806e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:40.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.681+0000 7ff34e400640 1 -- 172.21.15.143:0/53324061 >> 172.21.15.143:0/53324061 conn(0x7ff3480cbb40 msgr2=0x7ff3480cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:40.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.682+0000 7ff34e400640 1 -- 172.21.15.143:0/53324061 shutdown_connections 2024-01-16T15:00:40.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.682+0000 7ff34e400640 1 -- 172.21.15.143:0/53324061 wait complete. 2024-01-16T15:00:40.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.683+0000 7ff34e400640 1 Processor -- start 2024-01-16T15:00:40.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.683+0000 7ff34e400640 1 -- start start 2024-01-16T15:00:40.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.684+0000 7ff34e400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff34806e580 0x7ff34814b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:40.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.684+0000 7ff34e400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff3480b1250 0x7ff3480999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:40.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.684+0000 7ff34e400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff34815b260 0x7ff3481f3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:40.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.684+0000 7ff34e400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff348099f10 con 0x7ff34815b260 2024-01-16T15:00:40.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.684+0000 7ff34e400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3480b34d0 con 0x7ff34806e580 2024-01-16T15:00:40.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.684+0000 7ff34e400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff34809ce10 con 0x7ff3480b1250 2024-01-16T15:00:40.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.685+0000 7ff34da00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff34815b260 0x7ff3481f3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:40.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.685+0000 7ff34da00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff34815b260 0x7ff3481f3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43326/0 (socket says 172.21.15.143:43326) 2024-01-16T15:00:40.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.685+0000 7ff34da00640 1 -- 172.21.15.143:0/3156335856 learned_addr learned my addr 172.21.15.143:0/3156335856 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:40.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.685+0000 7ff347e00640 1 --2- 172.21.15.143:0/3156335856 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff3480b1250 0x7ff3480999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:40.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.685+0000 7ff34d000640 1 --2- 172.21.15.143:0/3156335856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff34806e580 0x7ff34814b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:40.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.685+0000 7ff34da00640 1 -- 172.21.15.143:0/3156335856 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff3480b1250 msgr2=0x7ff3480999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:40.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.685+0000 7ff34da00640 1 --2- 172.21.15.143:0/3156335856 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff3480b1250 0x7ff3480999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:40.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.685+0000 7ff34da00640 1 -- 172.21.15.143:0/3156335856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff34806e580 msgr2=0x7ff34814b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:40.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.685+0000 7ff34da00640 1 --2- 172.21.15.143:0/3156335856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff34806e580 0x7ff34814b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:40.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.685+0000 7ff34da00640 1 -- 172.21.15.143:0/3156335856 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff340009a70 con 0x7ff34815b260 2024-01-16T15:00:40.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.685+0000 7ff34da00640 1 --2- 172.21.15.143:0/3156335856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff34815b260 0x7ff3481f3100 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff340009d90 tx=0x7ff340004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:40.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.686+0000 7ff344c00640 1 -- 172.21.15.143:0/3156335856 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff340009450 con 0x7ff34815b260 2024-01-16T15:00:40.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.686+0000 7ff344c00640 1 -- 172.21.15.143:0/3156335856 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff340040450 con 0x7ff34815b260 2024-01-16T15:00:40.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.686+0000 7ff344c00640 1 -- 172.21.15.143:0/3156335856 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff340040800 con 0x7ff34815b260 2024-01-16T15:00:40.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.686+0000 7ff34e400640 1 -- 172.21.15.143:0/3156335856 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3480b3720 con 0x7ff34815b260 2024-01-16T15:00:40.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.686+0000 7ff34e400640 1 -- 172.21.15.143:0/3156335856 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3480b3ce0 con 0x7ff34815b260 2024-01-16T15:00:40.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.687+0000 7ff34e400640 1 -- 172.21.15.143:0/3156335856 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff34806e340 con 0x7ff34815b260 2024-01-16T15:00:40.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.688+0000 7ff344c00640 1 -- 172.21.15.143:0/3156335856 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7ff340040960 con 0x7ff34815b260 2024-01-16T15:00:40.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.689+0000 7ff344c00640 1 --2- 172.21.15.143:0/3156335856 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7ff32006b2c0 0x7ff32006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:40.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.689+0000 7ff344c00640 1 -- 172.21.15.143:0/3156335856 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff3400ac720 con 0x7ff34815b260 2024-01-16T15:00:40.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.689+0000 7ff34d000640 1 --2- 172.21.15.143:0/3156335856 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7ff32006b2c0 0x7ff32006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:40.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.690+0000 7ff34d000640 1 --2- 172.21.15.143:0/3156335856 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7ff32006b2c0 0x7ff32006d780 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff330006fd0 tx=0x7ff330008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:40.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.693+0000 7ff344c00640 1 -- 172.21.15.143:0/3156335856 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff340077620 con 0x7ff34815b260 2024-01-16T15:00:40.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.884+0000 7ff34e400640 1 -- 172.21.15.143:0/3156335856 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff34804e840 con 0x7ff34815b260 2024-01-16T15:00:40.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.884+0000 7ff344c00640 1 -- 172.21.15.143:0/3156335856 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff3400810a0 con 0x7ff34815b260 2024-01-16T15:00:40.864 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2024-01-16T15:00:40.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.887+0000 7ff34e400640 1 -- 172.21.15.143:0/3156335856 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7ff32006b2c0 msgr2=0x7ff32006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:40.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.887+0000 7ff34e400640 1 --2- 172.21.15.143:0/3156335856 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7ff32006b2c0 0x7ff32006d780 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff330006fd0 tx=0x7ff330008040 comp rx=0 tx=0).stop 2024-01-16T15:00:40.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.887+0000 7ff34e400640 1 -- 172.21.15.143:0/3156335856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff34815b260 msgr2=0x7ff3481f3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:40.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.887+0000 7ff34e400640 1 --2- 172.21.15.143:0/3156335856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff34815b260 0x7ff3481f3100 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff340009d90 tx=0x7ff340004a40 comp rx=0 tx=0).stop 2024-01-16T15:00:40.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.887+0000 7ff34e400640 1 -- 172.21.15.143:0/3156335856 shutdown_connections 2024-01-16T15:00:40.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.887+0000 7ff34e400640 1 --2- 172.21.15.143:0/3156335856 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7ff32006b2c0 0x7ff32006d780 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:40.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.887+0000 7ff34e400640 1 --2- 172.21.15.143:0/3156335856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff34815b260 0x7ff3481f3100 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:40.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.887+0000 7ff34e400640 1 --2- 172.21.15.143:0/3156335856 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff3480b1250 0x7ff3480999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:40.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.887+0000 7ff34e400640 1 --2- 172.21.15.143:0/3156335856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff34806e580 0x7ff34814b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:40.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.887+0000 7ff34e400640 1 -- 172.21.15.143:0/3156335856 >> 172.21.15.143:0/3156335856 conn(0x7ff3480cbb40 msgr2=0x7ff3480ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:40.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.887+0000 7ff34e400640 1 -- 172.21.15.143:0/3156335856 shutdown_connections 2024-01-16T15:00:40.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:40.888+0000 7ff34e400640 1 -- 172.21.15.143:0/3156335856 wait complete. 2024-01-16T15:00:40.886 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-16T15:00:41.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.123+0000 7f5cf3c00640 1 Processor -- start 2024-01-16T15:00:41.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.123+0000 7f5cf3c00640 1 -- start start 2024-01-16T15:00:41.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.124+0000 7f5cf3c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5cec15d7f0 0x7f5cec0c5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:41.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.125+0000 7f5cf3c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cec06d600 0x7f5cec0cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:41.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.125+0000 7f5cf3c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cec15e400 con 0x7f5cec0c4220 2024-01-16T15:00:41.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.125+0000 7f5cf3c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cec0ce880 con 0x7f5cec15d7f0 2024-01-16T15:00:41.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.125+0000 7f5cf3c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec0c5ed0 con 0x7f5cec06d600 2024-01-16T15:00:41.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.125+0000 7f5cf1200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5cec15d7f0 0x7f5cec0c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:41.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.125+0000 7f5cf1200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5cec15d7f0 0x7f5cec0c5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43328/0 (socket says 172.21.15.143:43328) 2024-01-16T15:00:41.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.125+0000 7f5cf1200640 1 -- 172.21.15.143:0/1706393420 learned_addr learned my addr 172.21.15.143:0/1706393420 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:41.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.125+0000 7f5cf1200640 1 -- 172.21.15.143:0/1706393420 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cec06d600 msgr2=0x7f5cec0cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:41.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.125+0000 7f5cf1200640 1 --2- 172.21.15.143:0/1706393420 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cec06d600 0x7f5cec0cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:41.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.125+0000 7f5cf1200640 1 -- 172.21.15.143:0/1706393420 >> v1:172.21.15.204:6789/0 conn(0x7f5cec0c4220 legacy=0x7f5cec0cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:41.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.125+0000 7f5cf1200640 1 -- 172.21.15.143:0/1706393420 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cec0c46e0 con 0x7f5cec15d7f0 2024-01-16T15:00:41.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.126+0000 7f5cf1200640 1 --2- 172.21.15.143:0/1706393420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cec15d7f0 0x7f5cec0c5990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5cec06ea50 tx=0x7f5cdc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b196607d49000f3 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:41.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.127+0000 7f5ceb400640 1 -- 172.21.15.143:0/1706393420 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc031640 con 0x7f5cec15d7f0 2024-01-16T15:00:41.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.127+0000 7f5ceb400640 1 -- 172.21.15.143:0/1706393420 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cdc0317a0 con 0x7f5cec15d7f0 2024-01-16T15:00:41.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.127+0000 7f5ceb400640 1 -- 172.21.15.143:0/1706393420 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc031b30 con 0x7f5cec15d7f0 2024-01-16T15:00:41.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.127+0000 7f5cf3c00640 1 -- 172.21.15.143:0/1706393420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cec15d7f0 msgr2=0x7f5cec0c5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:41.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.127+0000 7f5cf3c00640 1 --2- 172.21.15.143:0/1706393420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cec15d7f0 0x7f5cec0c5990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5cec06ea50 tx=0x7f5cdc02b8f0 comp rx=0 tx=0).stop 2024-01-16T15:00:41.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.127+0000 7f5cf3c00640 1 -- 172.21.15.143:0/1706393420 shutdown_connections 2024-01-16T15:00:41.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.127+0000 7f5cf3c00640 1 --2- 172.21.15.143:0/1706393420 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cec06d600 0x7f5cec0cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:41.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.128+0000 7f5cf3c00640 1 --2- 172.21.15.143:0/1706393420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cec15d7f0 0x7f5cec0c5990 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:41.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.128+0000 7f5cf3c00640 1 -- 172.21.15.143:0/1706393420 >> 172.21.15.143:0/1706393420 conn(0x7f5cec0b0c50 msgr2=0x7f5cec0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:41.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.128+0000 7f5cf3c00640 1 -- 172.21.15.143:0/1706393420 shutdown_connections 2024-01-16T15:00:41.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.128+0000 7f5cf3c00640 1 -- 172.21.15.143:0/1706393420 wait complete. 2024-01-16T15:00:41.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.129+0000 7f5cf3c00640 1 Processor -- start 2024-01-16T15:00:41.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.129+0000 7f5cf3c00640 1 -- start start 2024-01-16T15:00:41.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.130+0000 7f5cf3c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cec06d600 0x7f5cec1571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:41.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.130+0000 7f5cf3c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cec0c4220 0x7f5cec1f6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:41.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.130+0000 7f5cf3c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cec15d7f0 0x7f5cec1f6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:41.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.130+0000 7f5cf3c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec15d210 con 0x7f5cec06d600 2024-01-16T15:00:41.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.130+0000 7f5cf3c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec15cf30 con 0x7f5cec15d7f0 2024-01-16T15:00:41.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.130+0000 7f5cf3c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec070710 con 0x7f5cec0c4220 2024-01-16T15:00:41.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.131+0000 7f5cf1200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cec06d600 0x7f5cec1571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:41.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.131+0000 7f5cf1200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cec06d600 0x7f5cec1571f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43338/0 (socket says 172.21.15.143:43338) 2024-01-16T15:00:41.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.131+0000 7f5cf1200640 1 -- 172.21.15.143:0/2860684767 learned_addr learned my addr 172.21.15.143:0/2860684767 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:41.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.131+0000 7f5cf1200640 1 -- 172.21.15.143:0/2860684767 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cec0c4220 msgr2=0x7f5cec1f6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:41.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.131+0000 7f5cf1200640 1 --2- 172.21.15.143:0/2860684767 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cec0c4220 0x7f5cec1f6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:41.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.131+0000 7f5cf1200640 1 -- 172.21.15.143:0/2860684767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cec15d7f0 msgr2=0x7f5cec1f6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:41.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.131+0000 7f5cf1c00640 1 --2- 172.21.15.143:0/2860684767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cec15d7f0 0x7f5cec1f6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:41.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.131+0000 7f5cf1200640 1 --2- 172.21.15.143:0/2860684767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cec15d7f0 0x7f5cec1f6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:41.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.131+0000 7f5cf1200640 1 -- 172.21.15.143:0/2860684767 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cdc0094d0 con 0x7f5cec06d600 2024-01-16T15:00:41.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.131+0000 7f5cf1200640 1 --2- 172.21.15.143:0/2860684767 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cec06d600 0x7f5cec1571f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc000c00 tx=0x7f5cdc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:41.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.132+0000 7f5ce8c00640 1 -- 172.21.15.143:0/2860684767 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc03ac40 con 0x7f5cec06d600 2024-01-16T15:00:41.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.132+0000 7f5ce8c00640 1 -- 172.21.15.143:0/2860684767 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cdc03adc0 con 0x7f5cec06d600 2024-01-16T15:00:41.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.132+0000 7f5ce8c00640 1 -- 172.21.15.143:0/2860684767 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc044630 con 0x7f5cec06d600 2024-01-16T15:00:41.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.132+0000 7f5cf3c00640 1 -- 172.21.15.143:0/2860684767 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cec0709f0 con 0x7f5cec06d600 2024-01-16T15:00:41.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.132+0000 7f5cf3c00640 1 -- 172.21.15.143:0/2860684767 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cec1f7460 con 0x7f5cec06d600 2024-01-16T15:00:41.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.133+0000 7f5cf3c00640 1 -- 172.21.15.143:0/2860684767 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cec04e8b0 con 0x7f5cec06d600 2024-01-16T15:00:41.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.134+0000 7f5ce8c00640 1 -- 172.21.15.143:0/2860684767 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f5cdc02f050 con 0x7f5cec06d600 2024-01-16T15:00:41.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.135+0000 7f5ce8c00640 1 --2- 172.21.15.143:0/2860684767 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f5cc406b2c0 0x7f5cc406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:41.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.135+0000 7f5ce8c00640 1 -- 172.21.15.143:0/2860684767 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5cdc0abb20 con 0x7f5cec06d600 2024-01-16T15:00:41.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.135+0000 7f5cebe00640 1 --2- 172.21.15.143:0/2860684767 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f5cc406b2c0 0x7f5cc406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:41.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.136+0000 7f5cebe00640 1 --2- 172.21.15.143:0/2860684767 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f5cc406b2c0 0x7f5cc406d780 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0afea0 tx=0x7f5cd4007990 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:41.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.139+0000 7f5ce8c00640 1 -- 172.21.15.143:0/2860684767 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5cdc076a20 con 0x7f5cec06d600 2024-01-16T15:00:41.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.330+0000 7f5cf3c00640 1 -- 172.21.15.143:0/2860684767 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5cec1584e0 con 0x7f5cec06d600 2024-01-16T15:00:41.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.330+0000 7f5ce8c00640 1 -- 172.21.15.143:0/2860684767 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5cdc031e60 con 0x7f5cec06d600 2024-01-16T15:00:41.311 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2024-01-16T15:00:41.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.333+0000 7f5cf3c00640 1 -- 172.21.15.143:0/2860684767 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f5cc406b2c0 msgr2=0x7f5cc406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:41.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.333+0000 7f5cf3c00640 1 --2- 172.21.15.143:0/2860684767 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f5cc406b2c0 0x7f5cc406d780 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0afea0 tx=0x7f5cd4007990 comp rx=0 tx=0).stop 2024-01-16T15:00:41.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.333+0000 7f5cf3c00640 1 -- 172.21.15.143:0/2860684767 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cec06d600 msgr2=0x7f5cec1571f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:41.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.333+0000 7f5cf3c00640 1 --2- 172.21.15.143:0/2860684767 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cec06d600 0x7f5cec1571f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc000c00 tx=0x7f5cdc0046c0 comp rx=0 tx=0).stop 2024-01-16T15:00:41.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.333+0000 7f5cf3c00640 1 -- 172.21.15.143:0/2860684767 shutdown_connections 2024-01-16T15:00:41.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.333+0000 7f5cf3c00640 1 --2- 172.21.15.143:0/2860684767 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f5cc406b2c0 0x7f5cc406d780 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:41.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.333+0000 7f5cf3c00640 1 --2- 172.21.15.143:0/2860684767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cec15d7f0 0x7f5cec1f6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:41.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.333+0000 7f5cf3c00640 1 --2- 172.21.15.143:0/2860684767 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cec0c4220 0x7f5cec1f6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:41.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.333+0000 7f5cf3c00640 1 --2- 172.21.15.143:0/2860684767 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cec06d600 0x7f5cec1571f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:41.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.333+0000 7f5cf3c00640 1 -- 172.21.15.143:0/2860684767 >> 172.21.15.143:0/2860684767 conn(0x7f5cec0b0c50 msgr2=0x7f5cec0ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:41.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.333+0000 7f5cf3c00640 1 -- 172.21.15.143:0/2860684767 shutdown_connections 2024-01-16T15:00:41.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.333+0000 7f5cf3c00640 1 -- 172.21.15.143:0/2860684767 wait complete. 2024-01-16T15:00:41.395 INFO:tasks.cephfs_test_runner:Starting test: test_progress_bar_for_1_clone (tasks.cephfs.test_volumes.TestCloneProgress) Test that when one clone is in progress, one progress bar is printed 2024-01-16T15:00:41.395 INFO:tasks.cephfs_test_runner:test_progress_bar_for_1_clone (tasks.cephfs.test_volumes.TestCloneProgress) 2024-01-16T15:00:41.395 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T15:00:41.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.624+0000 7f34adc00640 1 Processor -- start 2024-01-16T15:00:41.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.624+0000 7f34adc00640 1 -- start start 2024-01-16T15:00:41.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.625+0000 7f34adc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34a80ce6c0 0x7f34a80cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:41.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.625+0000 7f34adc00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34a80cece0 con 0x7f34a80cf450 2024-01-16T15:00:41.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.625+0000 7f34adc00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34a806d040 con 0x7f34a815c7f0 2024-01-16T15:00:41.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.625+0000 7f34adc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34a80cfe40 con 0x7f34a80ce6c0 2024-01-16T15:00:41.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.625+0000 7f349de00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f34a80cf450 0x7f34a80cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49278/0 (socket says 172.21.15.143:49278) 2024-01-16T15:00:41.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.625+0000 7f349de00640 1 -- 172.21.15.143:0/538400766 learned_addr learned my addr 172.21.15.143:0/538400766 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:41.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.626+0000 7f34a6000640 1 -- 172.21.15.143:0/538400766 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3583591217 0 0) 0x7f34a80cece0 con 0x7f34a80cf450 2024-01-16T15:00:41.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.626+0000 7f34a6000640 1 -- 172.21.15.143:0/538400766 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3488003520 con 0x7f34a80cf450 2024-01-16T15:00:41.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.626+0000 7f34a7400640 1 --2- 172.21.15.143:0/538400766 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34a80ce6c0 0x7f34a80cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:41.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.626+0000 7f34a6000640 1 -- 172.21.15.143:0/538400766 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2457088645 0 0) 0x7f3488003520 con 0x7f34a80cf450 2024-01-16T15:00:41.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.626+0000 7f34a6000640 1 -- 172.21.15.143:0/538400766 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34a80ce6c0 msgr2=0x7f34a80cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:41.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.626+0000 7f34a6000640 1 --2- 172.21.15.143:0/538400766 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34a80ce6c0 0x7f34a80cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:41.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.626+0000 7f34a6000640 1 -- 172.21.15.143:0/538400766 >> v1:172.21.15.204:6789/0 conn(0x7f34a815c7f0 legacy=0x7f34a806db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:41.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.626+0000 7f34a6000640 1 -- 172.21.15.143:0/538400766 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a806d320 con 0x7f34a80cf450 2024-01-16T15:00:41.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.626+0000 7f34a6000640 1 -- 172.21.15.143:0/538400766 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7f3490001670 con 0x7f34a80cf450 2024-01-16T15:00:41.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.626+0000 7f34a6000640 1 -- 172.21.15.143:0/538400766 >> v1:172.21.15.143:6789/0 conn(0x7f34a80cf450 legacy=0x7f34a80cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:41.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.626+0000 7f34a6000640 1 --2- 172.21.15.143:0/538400766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34880039a0 0x7f3488003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:41.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.626+0000 7f34a6000640 1 --2- 172.21.15.143:0/538400766 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34880047c0 0x7f3488004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:41.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.626+0000 7f34a6000640 1 --2- 172.21.15.143:0/538400766 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34a80ce6c0 0x7f34880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:41.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.626+0000 7f34a6000640 1 -- 172.21.15.143:0/538400766 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a80cfe40 con 0x7f34880047c0 2024-01-16T15:00:41.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.626+0000 7f34a6000640 1 -- 172.21.15.143:0/538400766 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348800bab0 con 0x7f34880039a0 2024-01-16T15:00:41.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.626+0000 7f34a6000640 1 -- 172.21.15.143:0/538400766 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f348800bc70 con 0x7f34a80ce6c0 2024-01-16T15:00:41.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.627+0000 7f349de00640 1 --2- 172.21.15.143:0/538400766 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34880047c0 0x7f3488004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:41.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.627+0000 7f349de00640 1 -- 172.21.15.143:0/538400766 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34a80ce6c0 msgr2=0x7f34880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:41.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.627+0000 7f349de00640 1 --2- 172.21.15.143:0/538400766 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34a80ce6c0 0x7f34880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:41.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.627+0000 7f349de00640 1 -- 172.21.15.143:0/538400766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34880039a0 msgr2=0x7f3488003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:41.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.627+0000 7f349de00640 1 --2- 172.21.15.143:0/538400766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34880039a0 0x7f3488003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:41.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.627+0000 7f349de00640 1 -- 172.21.15.143:0/538400766 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3488003520 con 0x7f34880047c0 2024-01-16T15:00:41.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.628+0000 7f349de00640 1 --2- 172.21.15.143:0/538400766 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34880047c0 0x7f3488004c40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f349000c240 tx=0x7f34880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=847b4ef0e50c3674 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:41.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.628+0000 7f34a6000640 1 -- 172.21.15.143:0/538400766 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3490005ad0 con 0x7f34880047c0 2024-01-16T15:00:41.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.628+0000 7f34a6000640 1 -- 172.21.15.143:0/538400766 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3490005c30 con 0x7f34880047c0 2024-01-16T15:00:41.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.628+0000 7f34a6000640 1 -- 172.21.15.143:0/538400766 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349002c8b0 con 0x7f34880047c0 2024-01-16T15:00:41.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.629+0000 7f34adc00640 1 -- 172.21.15.143:0/538400766 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34880047c0 msgr2=0x7f3488004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:41.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.629+0000 7f34adc00640 1 --2- 172.21.15.143:0/538400766 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34880047c0 0x7f3488004c40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f349000c240 tx=0x7f34880057f0 comp rx=0 tx=0).stop 2024-01-16T15:00:41.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.629+0000 7f34a7400640 1 -- 172.21.15.143:0/538400766 reap_dead start 2024-01-16T15:00:41.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.629+0000 7f34adc00640 1 -- 172.21.15.143:0/538400766 shutdown_connections 2024-01-16T15:00:41.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.629+0000 7f34adc00640 1 -- 172.21.15.143:0/538400766 >> 172.21.15.143:0/538400766 conn(0x7f34a806a920 msgr2=0x7f34a806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:41.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.629+0000 7f34adc00640 1 -- 172.21.15.143:0/538400766 shutdown_connections 2024-01-16T15:00:41.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.629+0000 7f34adc00640 1 -- 172.21.15.143:0/538400766 wait complete. 2024-01-16T15:00:41.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.631+0000 7f34adc00640 1 Processor -- start 2024-01-16T15:00:41.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.631+0000 7f34adc00640 1 -- start start 2024-01-16T15:00:41.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.631+0000 7f34adc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34880047c0 0x7f34a81f3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:41.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.632+0000 7f34adc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34a81f5d80 0x7f34a8071b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:41.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.632+0000 7f34adc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34a80720d0 0x7f34a81f6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:41.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.632+0000 7f34adc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a8072550 con 0x7f34a80720d0 2024-01-16T15:00:41.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.632+0000 7f34adc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a81f7360 con 0x7f34a81f5d80 2024-01-16T15:00:41.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.632+0000 7f34adc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34a80b2540 con 0x7f34880047c0 2024-01-16T15:00:41.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.632+0000 7f34a7400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34a80720d0 0x7f34a81f6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:41.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.632+0000 7f34a7400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34a80720d0 0x7f34a81f6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43354/0 (socket says 172.21.15.143:43354) 2024-01-16T15:00:41.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.632+0000 7f34a7400640 1 -- 172.21.15.143:0/3618527524 learned_addr learned my addr 172.21.15.143:0/3618527524 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:41.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.632+0000 7f34a6a00640 1 --2- 172.21.15.143:0/3618527524 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34880047c0 0x7f34a81f3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:41.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.632+0000 7f349de00640 1 --2- 172.21.15.143:0/3618527524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34a81f5d80 0x7f34a8071b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:41.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.632+0000 7f34a7400640 1 -- 172.21.15.143:0/3618527524 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34880047c0 msgr2=0x7f34a81f3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:41.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.632+0000 7f34a7400640 1 --2- 172.21.15.143:0/3618527524 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34880047c0 0x7f34a81f3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:41.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.632+0000 7f34a7400640 1 -- 172.21.15.143:0/3618527524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34a81f5d80 msgr2=0x7f34a8071b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:41.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.632+0000 7f34a7400640 1 --2- 172.21.15.143:0/3618527524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34a81f5d80 0x7f34a8071b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:41.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.632+0000 7f34a7400640 1 -- 172.21.15.143:0/3618527524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a80b2820 con 0x7f34a80720d0 2024-01-16T15:00:41.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.632+0000 7f349de00640 1 --2- 172.21.15.143:0/3618527524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34a81f5d80 0x7f34a8071b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:41.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.633+0000 7f34a7400640 1 --2- 172.21.15.143:0/3618527524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34a80720d0 0x7f34a81f6610 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f34a80b5270 tx=0x7f34a000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:41.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.633+0000 7f349f400640 1 -- 172.21.15.143:0/3618527524 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a000ed90 con 0x7f34a80720d0 2024-01-16T15:00:41.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.633+0000 7f349f400640 1 -- 172.21.15.143:0/3618527524 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34a0003990 con 0x7f34a80720d0 2024-01-16T15:00:41.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.633+0000 7f349f400640 1 -- 172.21.15.143:0/3618527524 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a0003d40 con 0x7f34a80720d0 2024-01-16T15:00:41.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.633+0000 7f34adc00640 1 -- 172.21.15.143:0/3618527524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34a80b2b00 con 0x7f34a80720d0 2024-01-16T15:00:41.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.633+0000 7f34adc00640 1 -- 172.21.15.143:0/3618527524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34a81f6200 con 0x7f34a80720d0 2024-01-16T15:00:41.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.634+0000 7f34adc00640 1 -- 172.21.15.143:0/3618527524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34a80b3020 con 0x7f34a80720d0 2024-01-16T15:00:41.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.635+0000 7f349f400640 1 -- 172.21.15.143:0/3618527524 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f34a0004030 con 0x7f34a80720d0 2024-01-16T15:00:41.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.636+0000 7f349f400640 1 --2- 172.21.15.143:0/3618527524 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f348006b2f0 0x7f348006d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:41.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.636+0000 7f349f400640 1 -- 172.21.15.143:0/3618527524 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f34a0089750 con 0x7f34a80720d0 2024-01-16T15:00:41.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.636+0000 7f34a6a00640 1 --2- 172.21.15.143:0/3618527524 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f348006b2f0 0x7f348006d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:41.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.637+0000 7f34a6a00640 1 --2- 172.21.15.143:0/3618527524 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f348006b2f0 0x7f348006d7b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3498001690 tx=0x7f3498009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:41.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.640+0000 7f349f400640 1 -- 172.21.15.143:0/3618527524 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f34a80b3020 con 0x7f34a80720d0 2024-01-16T15:00:41.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.833+0000 7f34adc00640 1 -- 172.21.15.143:0/3618527524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34a80b3020 con 0x7f34a80720d0 2024-01-16T15:00:41.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.836+0000 7f349f400640 1 -- 172.21.15.143:0/3618527524 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16565 (secure 0 0 0) 0x7f34a80b3020 con 0x7f34a80720d0 2024-01-16T15:00:41.816 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:00:41.816 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0","created":"2024-01-16T14:59:38.532601+0000","modified":"2024-01-16T15:00:04.543620+0000","last_up_change":"2024-01-16T14:59:50.983699+0000","last_in_change":"2024-01-16T14:59:42.056297+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:59:51.727720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T15:00:01.010472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T15:00:02.041228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f4c8891d-ab29-472a-907d-8446f20f7012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6801","nonce":1569293626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6805","nonce":1569293626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6814","nonce":1569293626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6809","nonce":1569293626}]},"public_addr":"172.21.15.143:6801/1569293626","cluster_addr":"172.21.15.143:6805/1569293626","heartbeat_back_addr":"172.21.15.143:6814/1569293626","heartbeat_front_addr":"172.21.15.143:6809/1569293626","state":["exists","up"]},{"osd":1,"uuid":"c4a2788d-8b89-4992-82b2-7f4b9debd509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6804","nonce":1276841168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6808","nonce":1276841168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6813","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6815","nonce":1276841168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6811","nonce":1276841168}]},"public_addr":"172.21.15.143:6804/1276841168","cluster_addr":"172.21.15.143:6808/1276841168","heartbeat_back_addr":"172.21.15.143:6815/1276841168","heartbeat_front_addr":"172.21.15.143:6811/1276841168","state":["exists","up"]},{"osd":2,"uuid":"64645c1d-31b2-4794-b0f2-7ccf8df30393","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6817","nonce":1217670170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6819","nonce":1217670170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6823","nonce":1217670170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6821","nonce":1217670170}]},"public_addr":"172.21.15.143:6817/1217670170","cluster_addr":"172.21.15.143:6819/1217670170","heartbeat_back_addr":"172.21.15.143:6823/1217670170","heartbeat_front_addr":"172.21.15.143:6821/1217670170","state":["exists","up"]},{"osd":3,"uuid":"73674129-a84e-4d31-8247-7d0179ae6df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6825","nonce":3414188012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6827","nonce":3414188012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6831","nonce":3414188012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6829","nonce":3414188012}]},"public_addr":"172.21.15.143:6825/3414188012","cluster_addr":"172.21.15.143:6827/3414188012","heartbeat_back_addr":"172.21.15.143:6831/3414188012","heartbeat_front_addr":"172.21.15.143:6829/3414188012","state":["exists","up"]},{"osd":4,"uuid":"cea28c9b-2f1c-4087-b415-9960789131d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6811","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6812","nonce":1070255716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6816","nonce":1070255716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6823","nonce":1070255716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6820","nonce":1070255716}]},"public_addr":"172.21.15.204:6812/1070255716","cluster_addr":"172.21.15.204:6816/1070255716","heartbeat_back_addr":"172.21.15.204:6823/1070255716","heartbeat_front_addr":"172.21.15.204:6820/1070255716","state":["exists","up"]},{"osd":5,"uuid":"33b63560-5574-4e78-983f-b398d67a2cb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6809","nonce":3605417837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6813","nonce":3605417837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6819","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6821","nonce":3605417837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6817","nonce":3605417837}]},"public_addr":"172.21.15.204:6809/3605417837","cluster_addr":"172.21.15.204:6813/3605417837","heartbeat_back_addr":"172.21.15.204:6821/3605417837","heartbeat_front_addr":"172.21.15.204:6817/3605417837","state":["exists","up"]},{"osd":6,"uuid":"df397427-8940-4eee-9c79-eece669d735d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6801","nonce":1085000534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6803","nonce":1085000534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6808","nonce":1085000534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6805","nonce":1085000534}]},"public_addr":"172.21.15.204:6801/1085000534","cluster_addr":"172.21.15.204:6803/1085000534","heartbeat_back_addr":"172.21.15.204:6808/1085000534","heartbeat_front_addr":"172.21.15.204:6805/1085000534","state":["exists","up"]},{"osd":7,"uuid":"a2e5a15c-8312-405a-ac02-f7e1fc9df2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6825","nonce":1337220609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6827","nonce":1337220609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6831","nonce":1337220609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6829","nonce":1337220609}]},"public_addr":"172.21.15.204:6825/1337220609","cluster_addr":"172.21.15.204:6827/1337220609","heartbeat_back_addr":"172.21.15.204:6831/1337220609","heartbeat_front_addr":"172.21.15.204:6829/1337220609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:49.984507+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.395306+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.632998+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.373627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.342920+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.438366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.331926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.406657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T15:00:41.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.838+0000 7f34adc00640 1 -- 172.21.15.143:0/3618527524 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f348006b2f0 msgr2=0x7f348006d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:41.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.838+0000 7f34adc00640 1 --2- 172.21.15.143:0/3618527524 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f348006b2f0 0x7f348006d7b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3498001690 tx=0x7f3498009040 comp rx=0 tx=0).stop 2024-01-16T15:00:41.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.838+0000 7f34adc00640 1 -- 172.21.15.143:0/3618527524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34a80720d0 msgr2=0x7f34a81f6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:41.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.838+0000 7f34adc00640 1 --2- 172.21.15.143:0/3618527524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34a80720d0 0x7f34a81f6610 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f34a80b5270 tx=0x7f34a000e4b0 comp rx=0 tx=0).stop 2024-01-16T15:00:41.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.838+0000 7f34adc00640 1 -- 172.21.15.143:0/3618527524 shutdown_connections 2024-01-16T15:00:41.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.838+0000 7f34adc00640 1 --2- 172.21.15.143:0/3618527524 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f348006b2f0 0x7f348006d7b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:41.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.839+0000 7f34adc00640 1 --2- 172.21.15.143:0/3618527524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34a80720d0 0x7f34a81f6610 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:41.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.839+0000 7f34adc00640 1 --2- 172.21.15.143:0/3618527524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34a81f5d80 0x7f34a8071b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:41.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.839+0000 7f34adc00640 1 --2- 172.21.15.143:0/3618527524 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34880047c0 0x7f34a81f3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:41.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.839+0000 7f34adc00640 1 -- 172.21.15.143:0/3618527524 >> 172.21.15.143:0/3618527524 conn(0x7f34a806a920 msgr2=0x7f34a8154e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:41.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.839+0000 7f34adc00640 1 -- 172.21.15.143:0/3618527524 shutdown_connections 2024-01-16T15:00:41.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:41.839+0000 7f34adc00640 1 -- 172.21.15.143:0/3618527524 wait complete. 2024-01-16T15:00:41.838 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-16T14:59:51.727720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-16T15:00:01.010472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-16T15:00:02.041228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-16T15:00:41.838 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-16T15:00:42.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.070+0000 7f61cd600640 1 Processor -- start 2024-01-16T15:00:42.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.070+0000 7f61cd600640 1 -- start start 2024-01-16T15:00:42.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.071+0000 7f61cd600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f61c815a1c0 0x7f61c806d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:42.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.071+0000 7f61cd600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f61c80b0b40 0x7f61c80b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:42.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.071+0000 7f61cd600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c806d430 con 0x7f61c815ded0 2024-01-16T15:00:42.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.071+0000 7f61cd600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c815ea70 con 0x7f61c815a1c0 2024-01-16T15:00:42.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.071+0000 7f61cd600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c815ec50 con 0x7f61c80b0b40 2024-01-16T15:00:42.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.071+0000 7f61c7e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f61c815a1c0 0x7f61c806d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:42.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.072+0000 7f61c7e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f61c815a1c0 0x7f61c806d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43362/0 (socket says 172.21.15.143:43362) 2024-01-16T15:00:42.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.072+0000 7f61c7e00640 1 -- 172.21.15.143:0/3713579659 learned_addr learned my addr 172.21.15.143:0/3713579659 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:42.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.072+0000 7f61bf200640 1 --2- 172.21.15.143:0/3713579659 >> v2:172.21.15.204:3300/0 conn(0x7f61c80b0b40 0x7f61c80b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:42.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.072+0000 7f61c7e00640 1 -- 172.21.15.143:0/3713579659 >> v2:172.21.15.204:3300/0 conn(0x7f61c80b0b40 msgr2=0x7f61c80b0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:42.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.072+0000 7f61c7e00640 1 --2- 172.21.15.143:0/3713579659 >> v2:172.21.15.204:3300/0 conn(0x7f61c80b0b40 0x7f61c80b0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:42.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.072+0000 7f61c7e00640 1 -- 172.21.15.143:0/3713579659 >> v1:172.21.15.143:6789/0 conn(0x7f61c815ded0 legacy=0x7f61c815e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:42.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.072+0000 7f61c7e00640 1 -- 172.21.15.143:0/3713579659 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c80b14e0 con 0x7f61c815a1c0 2024-01-16T15:00:42.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.073+0000 7f61c7e00640 1 --2- 172.21.15.143:0/3713579659 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61c815a1c0 0x7f61c806d8c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f61c80cee20 tx=0x7f61b802dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87340482335a29ce server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:42.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.073+0000 7f61c7400640 1 -- 172.21.15.143:0/3713579659 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b802e960 con 0x7f61c815a1c0 2024-01-16T15:00:42.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.073+0000 7f61c7400640 1 -- 172.21.15.143:0/3713579659 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b802eac0 con 0x7f61c815a1c0 2024-01-16T15:00:42.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.073+0000 7f61c7400640 1 -- 172.21.15.143:0/3713579659 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b802ee50 con 0x7f61c815a1c0 2024-01-16T15:00:42.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.074+0000 7f61cd600640 1 -- 172.21.15.143:0/3713579659 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61c815a1c0 msgr2=0x7f61c806d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:42.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.074+0000 7f61cd600640 1 --2- 172.21.15.143:0/3713579659 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61c815a1c0 0x7f61c806d8c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f61c80cee20 tx=0x7f61b802dbb0 comp rx=0 tx=0).stop 2024-01-16T15:00:42.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.074+0000 7f61cd600640 1 -- 172.21.15.143:0/3713579659 shutdown_connections 2024-01-16T15:00:42.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.074+0000 7f61cd600640 1 --2- 172.21.15.143:0/3713579659 >> v2:172.21.15.204:3300/0 conn(0x7f61c80b0b40 0x7f61c80b0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:42.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.074+0000 7f61cd600640 1 --2- 172.21.15.143:0/3713579659 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61c815a1c0 0x7f61c806d8c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:42.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.074+0000 7f61cd600640 1 -- 172.21.15.143:0/3713579659 >> 172.21.15.143:0/3713579659 conn(0x7f61c80cbb40 msgr2=0x7f61c80cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:42.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.074+0000 7f61cd600640 1 -- 172.21.15.143:0/3713579659 shutdown_connections 2024-01-16T15:00:42.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.074+0000 7f61cd600640 1 -- 172.21.15.143:0/3713579659 wait complete. 2024-01-16T15:00:42.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.076+0000 7f61cd600640 1 Processor -- start 2024-01-16T15:00:42.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.076+0000 7f61cd600640 1 -- start start 2024-01-16T15:00:42.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.077+0000 7f61cd600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f61c80b0b40 0x7f61c80c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:42.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.077+0000 7f61cd600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61c815a1c0 0x7f61c814f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:42.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.077+0000 7f61cd600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61c815ded0 0x7f61c814f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:42.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.077+0000 7f61cd600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c80c6f70 con 0x7f61c815ded0 2024-01-16T15:00:42.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.077+0000 7f61cd600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c80c7250 con 0x7f61c815a1c0 2024-01-16T15:00:42.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.077+0000 7f61cd600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c80c7530 con 0x7f61c80b0b40 2024-01-16T15:00:42.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.077+0000 7f61ccc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61c815ded0 0x7f61c814f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:42.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.077+0000 7f61ccc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61c815ded0 0x7f61c814f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43370/0 (socket says 172.21.15.143:43370) 2024-01-16T15:00:42.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.077+0000 7f61ccc00640 1 -- 172.21.15.143:0/3057926696 learned_addr learned my addr 172.21.15.143:0/3057926696 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:42.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.078+0000 7f61c7e00640 1 --2- 172.21.15.143:0/3057926696 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f61c80b0b40 0x7f61c80c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:42.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.078+0000 7f61bf200640 1 --2- 172.21.15.143:0/3057926696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61c815a1c0 0x7f61c814f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:42.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.078+0000 7f61ccc00640 1 -- 172.21.15.143:0/3057926696 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f61c80b0b40 msgr2=0x7f61c80c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:42.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.078+0000 7f61ccc00640 1 --2- 172.21.15.143:0/3057926696 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f61c80b0b40 0x7f61c80c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:42.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.078+0000 7f61ccc00640 1 -- 172.21.15.143:0/3057926696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61c815a1c0 msgr2=0x7f61c814f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:42.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.078+0000 7f61ccc00640 1 --2- 172.21.15.143:0/3057926696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61c815a1c0 0x7f61c814f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:42.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.078+0000 7f61ccc00640 1 -- 172.21.15.143:0/3057926696 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b80094d0 con 0x7f61c815ded0 2024-01-16T15:00:42.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.078+0000 7f61ccc00640 1 --2- 172.21.15.143:0/3057926696 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61c815ded0 0x7f61c814f690 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f61c000c6b0 tx=0x7f61c000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:42.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.078+0000 7f61c4c00640 1 -- 172.21.15.143:0/3057926696 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c0003b70 con 0x7f61c815ded0 2024-01-16T15:00:42.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.079+0000 7f61c4c00640 1 -- 172.21.15.143:0/3057926696 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c0003cf0 con 0x7f61c815ded0 2024-01-16T15:00:42.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.079+0000 7f61c4c00640 1 -- 172.21.15.143:0/3057926696 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c0003ea0 con 0x7f61c815ded0 2024-01-16T15:00:42.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.079+0000 7f61cd600640 1 -- 172.21.15.143:0/3057926696 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c80c7810 con 0x7f61c815ded0 2024-01-16T15:00:42.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.079+0000 7f61cd600640 1 -- 172.21.15.143:0/3057926696 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c80c89a0 con 0x7f61c815ded0 2024-01-16T15:00:42.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.080+0000 7f61cd600640 1 -- 172.21.15.143:0/3057926696 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61c804e840 con 0x7f61c815ded0 2024-01-16T15:00:42.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.081+0000 7f61c4c00640 1 -- 172.21.15.143:0/3057926696 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f61c0005310 con 0x7f61c815ded0 2024-01-16T15:00:42.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.081+0000 7f61c4c00640 1 --2- 172.21.15.143:0/3057926696 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f61a006b2c0 0x7f61a006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:42.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.081+0000 7f61c4c00640 1 -- 172.21.15.143:0/3057926696 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f61c0089a20 con 0x7f61c815ded0 2024-01-16T15:00:42.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.082+0000 7f61c7e00640 1 --2- 172.21.15.143:0/3057926696 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f61a006b2c0 0x7f61a006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:42.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.082+0000 7f61c7e00640 1 --2- 172.21.15.143:0/3057926696 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f61a006b2c0 0x7f61a006d780 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f61b8038000 tx=0x7f61b8032000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:42.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.086+0000 7f61c4c00640 1 -- 172.21.15.143:0/3057926696 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f61c00547f0 con 0x7f61c815ded0 2024-01-16T15:00:42.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.276+0000 7f61cd600640 1 -- 172.21.15.143:0/3057926696 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f61c815e670 con 0x7f61c815ded0 2024-01-16T15:00:42.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.277+0000 7f61c4c00640 1 -- 172.21.15.143:0/3057926696 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f61c0057e10 con 0x7f61c815ded0 2024-01-16T15:00:42.257 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-16T15:00:42.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.280+0000 7f61cd600640 1 -- 172.21.15.143:0/3057926696 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f61a006b2c0 msgr2=0x7f61a006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:42.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.280+0000 7f61cd600640 1 --2- 172.21.15.143:0/3057926696 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f61a006b2c0 0x7f61a006d780 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f61b8038000 tx=0x7f61b8032000 comp rx=0 tx=0).stop 2024-01-16T15:00:42.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.280+0000 7f61cd600640 1 -- 172.21.15.143:0/3057926696 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61c815ded0 msgr2=0x7f61c814f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:42.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.280+0000 7f61cd600640 1 --2- 172.21.15.143:0/3057926696 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61c815ded0 0x7f61c814f690 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f61c000c6b0 tx=0x7f61c000cb70 comp rx=0 tx=0).stop 2024-01-16T15:00:42.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.280+0000 7f61cd600640 1 -- 172.21.15.143:0/3057926696 shutdown_connections 2024-01-16T15:00:42.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.280+0000 7f61cd600640 1 --2- 172.21.15.143:0/3057926696 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f61a006b2c0 0x7f61a006d780 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:42.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.280+0000 7f61cd600640 1 --2- 172.21.15.143:0/3057926696 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61c815ded0 0x7f61c814f690 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:42.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.280+0000 7f61cd600640 1 --2- 172.21.15.143:0/3057926696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61c815a1c0 0x7f61c814f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:42.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.280+0000 7f61cd600640 1 --2- 172.21.15.143:0/3057926696 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f61c80b0b40 0x7f61c80c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:42.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.280+0000 7f61cd600640 1 -- 172.21.15.143:0/3057926696 >> 172.21.15.143:0/3057926696 conn(0x7f61c80cbb40 msgr2=0x7f61c80cfa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:42.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.280+0000 7f61cd600640 1 -- 172.21.15.143:0/3057926696 shutdown_connections 2024-01-16T15:00:42.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.280+0000 7f61cd600640 1 -- 172.21.15.143:0/3057926696 wait complete. 2024-01-16T15:00:42.278 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-16T15:00:42.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.503+0000 7fc667e00640 1 Processor -- start 2024-01-16T15:00:42.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.503+0000 7fc667e00640 1 -- start start 2024-01-16T15:00:42.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.504+0000 7fc667e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc66815b260 0x7fc66806e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:42.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.504+0000 7fc667e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6680b1250 0x7fc66806c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:42.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.504+0000 7fc667e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc66815ed20 con 0x7fc66806e580 2024-01-16T15:00:42.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.505+0000 7fc667e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc66815b640 con 0x7fc66815b260 2024-01-16T15:00:42.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.505+0000 7fc667e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc66815f8a0 con 0x7fc6680b1250 2024-01-16T15:00:42.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.505+0000 7fc666000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc66806e580 0x7fc6680b0b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49302/0 (socket says 172.21.15.143:49302) 2024-01-16T15:00:42.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.505+0000 7fc666000640 1 -- 172.21.15.143:0/2631636926 learned_addr learned my addr 172.21.15.143:0/2631636926 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:42.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.505+0000 7fc667400640 1 --2- 172.21.15.143:0/2631636926 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6680b1250 0x7fc66806c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:42.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.505+0000 7fc666a00640 1 --2- 172.21.15.143:0/2631636926 >> v2:172.21.15.204:3300/0 conn(0x7fc66815b260 0x7fc66806e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:42.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.505+0000 7fc665600640 1 -- 172.21.15.143:0/2631636926 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 586877241 0 0) 0x7fc66815ed20 con 0x7fc66806e580 2024-01-16T15:00:42.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.505+0000 7fc665600640 1 -- 172.21.15.143:0/2631636926 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc648003520 con 0x7fc66806e580 2024-01-16T15:00:42.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.506+0000 7fc665600640 1 -- 172.21.15.143:0/2631636926 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1283560365 0 0) 0x7fc648003520 con 0x7fc66806e580 2024-01-16T15:00:42.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.506+0000 7fc665600640 1 -- 172.21.15.143:0/2631636926 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6680b1250 msgr2=0x7fc66806c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:42.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.506+0000 7fc665600640 1 --2- 172.21.15.143:0/2631636926 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6680b1250 0x7fc66806c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:42.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.506+0000 7fc665600640 1 -- 172.21.15.143:0/2631636926 >> v2:172.21.15.204:3300/0 conn(0x7fc66815b260 msgr2=0x7fc66806e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:42.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.506+0000 7fc665600640 1 --2- 172.21.15.143:0/2631636926 >> v2:172.21.15.204:3300/0 conn(0x7fc66815b260 0x7fc66806e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:42.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.506+0000 7fc665600640 1 -- 172.21.15.143:0/2631636926 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6680abe50 con 0x7fc66806e580 2024-01-16T15:00:42.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.506+0000 7fc665600640 1 -- 172.21.15.143:0/2631636926 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7fc650001670 con 0x7fc66806e580 2024-01-16T15:00:42.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.506+0000 7fc665600640 1 -- 172.21.15.143:0/2631636926 >> v1:172.21.15.143:6789/0 conn(0x7fc66806e580 legacy=0x7fc6680b0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:42.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.506+0000 7fc665600640 1 --2- 172.21.15.143:0/2631636926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc648003970 0x7fc648003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:42.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.506+0000 7fc665600640 1 --2- 172.21.15.143:0/2631636926 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6680b1250 0x7fc6480047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:42.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.506+0000 7fc665600640 1 --2- 172.21.15.143:0/2631636926 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc648004d00 0x7fc648005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:42.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.506+0000 7fc665600640 1 -- 172.21.15.143:0/2631636926 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66815b640 con 0x7fc648004d00 2024-01-16T15:00:42.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.506+0000 7fc665600640 1 -- 172.21.15.143:0/2631636926 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66815f8a0 con 0x7fc648003970 2024-01-16T15:00:42.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.506+0000 7fc665600640 1 -- 172.21.15.143:0/2631636926 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc64800ba30 con 0x7fc6680b1250 2024-01-16T15:00:42.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.506+0000 7fc667400640 1 --2- 172.21.15.143:0/2631636926 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc648004d00 0x7fc648005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:42.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.507+0000 7fc666000640 1 --2- 172.21.15.143:0/2631636926 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6680b1250 0x7fc6480047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:42.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.507+0000 7fc666a00640 1 --2- 172.21.15.143:0/2631636926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc648003970 0x7fc648003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:42.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.507+0000 7fc667400640 1 -- 172.21.15.143:0/2631636926 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6680b1250 msgr2=0x7fc6480047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:42.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.507+0000 7fc667400640 1 --2- 172.21.15.143:0/2631636926 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6680b1250 0x7fc6480047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:42.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.507+0000 7fc667400640 1 -- 172.21.15.143:0/2631636926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc648003970 msgr2=0x7fc648003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:42.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.507+0000 7fc667400640 1 --2- 172.21.15.143:0/2631636926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc648003970 0x7fc648003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:42.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.507+0000 7fc667400640 1 -- 172.21.15.143:0/2631636926 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc648003520 con 0x7fc648004d00 2024-01-16T15:00:42.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.508+0000 7fc667400640 1 --2- 172.21.15.143:0/2631636926 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc648004d00 0x7fc648005180 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc668070330 tx=0x7fc6480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2690cf4b4018ff5 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:42.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.508+0000 7fc665600640 1 -- 172.21.15.143:0/2631636926 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66002cb90 con 0x7fc648004d00 2024-01-16T15:00:42.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.508+0000 7fc665600640 1 -- 172.21.15.143:0/2631636926 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc66000f040 con 0x7fc648004d00 2024-01-16T15:00:42.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.508+0000 7fc665600640 1 -- 172.21.15.143:0/2631636926 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc660034440 con 0x7fc648004d00 2024-01-16T15:00:42.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.509+0000 7fc667e00640 1 -- 172.21.15.143:0/2631636926 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc648004d00 msgr2=0x7fc648005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:42.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.509+0000 7fc667e00640 1 --2- 172.21.15.143:0/2631636926 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc648004d00 0x7fc648005180 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc668070330 tx=0x7fc6480057f0 comp rx=0 tx=0).stop 2024-01-16T15:00:42.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.509+0000 7fc667400640 1 -- 172.21.15.143:0/2631636926 reap_dead start 2024-01-16T15:00:42.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.509+0000 7fc667e00640 1 -- 172.21.15.143:0/2631636926 shutdown_connections 2024-01-16T15:00:42.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.509+0000 7fc667e00640 1 -- 172.21.15.143:0/2631636926 >> 172.21.15.143:0/2631636926 conn(0x7fc6680cbb40 msgr2=0x7fc6680cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:42.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.509+0000 7fc667e00640 1 -- 172.21.15.143:0/2631636926 shutdown_connections 2024-01-16T15:00:42.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.509+0000 7fc667e00640 1 -- 172.21.15.143:0/2631636926 wait complete. 2024-01-16T15:00:42.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.511+0000 7fc667e00640 1 Processor -- start 2024-01-16T15:00:42.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.511+0000 7fc667e00640 1 -- start start 2024-01-16T15:00:42.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.512+0000 7fc667e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6681eff00 0x7fc6681f3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:42.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.512+0000 7fc667e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6680999d0 0x7fc6680b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:42.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.512+0000 7fc667e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6680b4830 0x7fc6681f6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:42.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.512+0000 7fc667e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6680b4cb0 con 0x7fc6680b4830 2024-01-16T15:00:42.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.512+0000 7fc667e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66809b9b0 con 0x7fc6680999d0 2024-01-16T15:00:42.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.512+0000 7fc667e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6681f3a50 con 0x7fc6681eff00 2024-01-16T15:00:42.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.512+0000 7fc667400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6680b4830 0x7fc6681f6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:42.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.512+0000 7fc667400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6680b4830 0x7fc6681f6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43390/0 (socket says 172.21.15.143:43390) 2024-01-16T15:00:42.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.512+0000 7fc667400640 1 -- 172.21.15.143:0/2370308381 learned_addr learned my addr 172.21.15.143:0/2370308381 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:42.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.513+0000 7fc666a00640 1 --2- 172.21.15.143:0/2370308381 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6681eff00 0x7fc6681f3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:42.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.513+0000 7fc667400640 1 -- 172.21.15.143:0/2370308381 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6681eff00 msgr2=0x7fc6681f3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:42.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.513+0000 7fc667400640 1 --2- 172.21.15.143:0/2370308381 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6681eff00 0x7fc6681f3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:42.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.513+0000 7fc666000640 1 --2- 172.21.15.143:0/2370308381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6680999d0 0x7fc6680b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:42.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.513+0000 7fc667400640 1 -- 172.21.15.143:0/2370308381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6680999d0 msgr2=0x7fc6680b42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:42.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.513+0000 7fc667400640 1 --2- 172.21.15.143:0/2370308381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6680999d0 0x7fc6680b42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:42.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.513+0000 7fc667400640 1 -- 172.21.15.143:0/2370308381 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc668099e90 con 0x7fc6680b4830 2024-01-16T15:00:42.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.513+0000 7fc666a00640 1 --2- 172.21.15.143:0/2370308381 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6681eff00 0x7fc6681f3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:42.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.513+0000 7fc667400640 1 --2- 172.21.15.143:0/2370308381 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6680b4830 0x7fc6681f6080 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc66000e230 tx=0x7fc660009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:42.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.513+0000 7fc65ea00640 1 -- 172.21.15.143:0/2370308381 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc660043070 con 0x7fc6680b4830 2024-01-16T15:00:42.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.513+0000 7fc65ea00640 1 -- 172.21.15.143:0/2370308381 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc660003990 con 0x7fc6680b4830 2024-01-16T15:00:42.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.513+0000 7fc65ea00640 1 -- 172.21.15.143:0/2370308381 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc660003d40 con 0x7fc6680b4830 2024-01-16T15:00:42.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.513+0000 7fc667e00640 1 -- 172.21.15.143:0/2370308381 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc66809a170 con 0x7fc6680b4830 2024-01-16T15:00:42.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.514+0000 7fc667e00640 1 -- 172.21.15.143:0/2370308381 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc66809a730 con 0x7fc6680b4830 2024-01-16T15:00:42.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.515+0000 7fc667e00640 1 -- 172.21.15.143:0/2370308381 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc66804e840 con 0x7fc6680b4830 2024-01-16T15:00:42.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.516+0000 7fc65ea00640 1 -- 172.21.15.143:0/2370308381 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fc660034880 con 0x7fc6680b4830 2024-01-16T15:00:42.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.517+0000 7fc65ea00640 1 --2- 172.21.15.143:0/2370308381 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fc64006b2c0 0x7fc64006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:42.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.517+0000 7fc65ea00640 1 -- 172.21.15.143:0/2370308381 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc6600ab280 con 0x7fc6680b4830 2024-01-16T15:00:42.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.517+0000 7fc666a00640 1 --2- 172.21.15.143:0/2370308381 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fc64006b2c0 0x7fc64006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:42.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.518+0000 7fc666a00640 1 --2- 172.21.15.143:0/2370308381 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fc64006b2c0 0x7fc64006d780 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc658009a00 tx=0x7fc658008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:42.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.521+0000 7fc65ea00640 1 -- 172.21.15.143:0/2370308381 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc660076180 con 0x7fc6680b4830 2024-01-16T15:00:42.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.714+0000 7fc667e00640 1 -- 172.21.15.143:0/2370308381 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc6680ce640 con 0x7fc6680b4830 2024-01-16T15:00:42.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.715+0000 7fc65ea00640 1 -- 172.21.15.143:0/2370308381 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc6600797a0 con 0x7fc6680b4830 2024-01-16T15:00:42.696 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2024-01-16T15:00:42.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.717+0000 7fc667e00640 1 -- 172.21.15.143:0/2370308381 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fc64006b2c0 msgr2=0x7fc64006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:42.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.717+0000 7fc667e00640 1 --2- 172.21.15.143:0/2370308381 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fc64006b2c0 0x7fc64006d780 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc658009a00 tx=0x7fc658008040 comp rx=0 tx=0).stop 2024-01-16T15:00:42.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.717+0000 7fc667e00640 1 -- 172.21.15.143:0/2370308381 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6680b4830 msgr2=0x7fc6681f6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:42.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.717+0000 7fc667e00640 1 --2- 172.21.15.143:0/2370308381 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6680b4830 0x7fc6681f6080 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc66000e230 tx=0x7fc660009750 comp rx=0 tx=0).stop 2024-01-16T15:00:42.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.718+0000 7fc667e00640 1 -- 172.21.15.143:0/2370308381 shutdown_connections 2024-01-16T15:00:42.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.718+0000 7fc667e00640 1 --2- 172.21.15.143:0/2370308381 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fc64006b2c0 0x7fc64006d780 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:42.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.718+0000 7fc667e00640 1 --2- 172.21.15.143:0/2370308381 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6680b4830 0x7fc6681f6080 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:42.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.718+0000 7fc667e00640 1 --2- 172.21.15.143:0/2370308381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6680999d0 0x7fc6680b42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:42.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.718+0000 7fc667e00640 1 --2- 172.21.15.143:0/2370308381 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6681eff00 0x7fc6681f3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:42.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.718+0000 7fc667e00640 1 -- 172.21.15.143:0/2370308381 >> 172.21.15.143:0/2370308381 conn(0x7fc6680cbb40 msgr2=0x7fc6680cf5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:42.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.718+0000 7fc667e00640 1 -- 172.21.15.143:0/2370308381 shutdown_connections 2024-01-16T15:00:42.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.718+0000 7fc667e00640 1 -- 172.21.15.143:0/2370308381 wait complete. 2024-01-16T15:00:42.715 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-16T15:00:42.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.952+0000 7fd566e00640 1 Processor -- start 2024-01-16T15:00:42.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.952+0000 7fd566e00640 1 -- start start 2024-01-16T15:00:42.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.953+0000 7fd566e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd56006de10 0x7fd56015e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:42.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.954+0000 7fd566e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5600c49f0 0x7fd5600705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:42.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.954+0000 7fd566e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd56015ec90 con 0x7fd560072160 2024-01-16T15:00:42.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.954+0000 7fd566e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5600cd530 con 0x7fd56006de10 2024-01-16T15:00:42.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.954+0000 7fd566e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5600c4ed0 con 0x7fd5600c49f0 2024-01-16T15:00:42.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.954+0000 7fd55f400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd56006de10 0x7fd56015e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:42.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.954+0000 7fd55f400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd56006de10 0x7fd56015e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43406/0 (socket says 172.21.15.143:43406) 2024-01-16T15:00:42.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.954+0000 7fd55f400640 1 -- 172.21.15.143:0/3464872187 learned_addr learned my addr 172.21.15.143:0/3464872187 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:42.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.954+0000 7fd55f400640 1 -- 172.21.15.143:0/3464872187 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5600c49f0 msgr2=0x7fd5600705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:42.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.954+0000 7fd55f400640 1 --2- 172.21.15.143:0/3464872187 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5600c49f0 0x7fd5600705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:42.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.954+0000 7fd55f400640 1 -- 172.21.15.143:0/3464872187 >> v1:172.21.15.204:6789/0 conn(0x7fd560072160 legacy=0x7fd56006d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:42.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.954+0000 7fd55f400640 1 -- 172.21.15.143:0/3464872187 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5600c51b0 con 0x7fd56006de10 2024-01-16T15:00:42.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.955+0000 7fd55f400640 1 --2- 172.21.15.143:0/3464872187 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd56006de10 0x7fd56015e750 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd54800bb10 tx=0x7fd54802e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40448793a530f3a8 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:42.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.956+0000 7fd55ea00640 1 -- 172.21.15.143:0/3464872187 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54802ee90 con 0x7fd56006de10 2024-01-16T15:00:42.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.956+0000 7fd55ea00640 1 -- 172.21.15.143:0/3464872187 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd548036440 con 0x7fd56006de10 2024-01-16T15:00:42.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.956+0000 7fd55ea00640 1 -- 172.21.15.143:0/3464872187 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5480367d0 con 0x7fd56006de10 2024-01-16T15:00:42.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.956+0000 7fd566e00640 1 -- 172.21.15.143:0/3464872187 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd56006de10 msgr2=0x7fd56015e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:42.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.956+0000 7fd566e00640 1 --2- 172.21.15.143:0/3464872187 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd56006de10 0x7fd56015e750 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd54800bb10 tx=0x7fd54802e0e0 comp rx=0 tx=0).stop 2024-01-16T15:00:42.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.956+0000 7fd566e00640 1 -- 172.21.15.143:0/3464872187 shutdown_connections 2024-01-16T15:00:42.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.956+0000 7fd566e00640 1 --2- 172.21.15.143:0/3464872187 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5600c49f0 0x7fd5600705c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:42.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.956+0000 7fd566e00640 1 --2- 172.21.15.143:0/3464872187 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd56006de10 0x7fd56015e750 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:42.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.956+0000 7fd566e00640 1 -- 172.21.15.143:0/3464872187 >> 172.21.15.143:0/3464872187 conn(0x7fd5600b0c50 msgr2=0x7fd5600b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:42.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.956+0000 7fd566e00640 1 -- 172.21.15.143:0/3464872187 shutdown_connections 2024-01-16T15:00:42.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.957+0000 7fd566e00640 1 -- 172.21.15.143:0/3464872187 wait complete. 2024-01-16T15:00:42.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.958+0000 7fd566e00640 1 Processor -- start 2024-01-16T15:00:42.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.958+0000 7fd566e00640 1 -- start start 2024-01-16T15:00:42.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.959+0000 7fd566e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd56006de10 0x7fd5600c60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:42.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.959+0000 7fd566e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd560072160 0x7fd56009a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:42.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.959+0000 7fd566e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5600c49f0 0x7fd56009c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:42.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.959+0000 7fd566e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56015b620 con 0x7fd5600c49f0 2024-01-16T15:00:42.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.959+0000 7fd566e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56015b8c0 con 0x7fd560072160 2024-01-16T15:00:42.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.959+0000 7fd566e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd56015bba0 con 0x7fd56006de10 2024-01-16T15:00:42.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.959+0000 7fd564e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5600c49f0 0x7fd56009c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:42.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.960+0000 7fd564e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5600c49f0 0x7fd56009c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43418/0 (socket says 172.21.15.143:43418) 2024-01-16T15:00:42.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.960+0000 7fd564e00640 1 -- 172.21.15.143:0/382041297 learned_addr learned my addr 172.21.15.143:0/382041297 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:42.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.960+0000 7fd55f400640 1 --2- 172.21.15.143:0/382041297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd560072160 0x7fd56009a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:42.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.960+0000 7fd55fe00640 1 --2- 172.21.15.143:0/382041297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd56006de10 0x7fd5600c60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:42.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.960+0000 7fd564e00640 1 -- 172.21.15.143:0/382041297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd56006de10 msgr2=0x7fd5600c60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:42.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.960+0000 7fd564e00640 1 --2- 172.21.15.143:0/382041297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd56006de10 0x7fd5600c60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:42.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.960+0000 7fd564e00640 1 -- 172.21.15.143:0/382041297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd560072160 msgr2=0x7fd56009a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:42.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.960+0000 7fd564e00640 1 --2- 172.21.15.143:0/382041297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd560072160 0x7fd56009a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:42.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.960+0000 7fd564e00640 1 -- 172.21.15.143:0/382041297 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd54800b770 con 0x7fd5600c49f0 2024-01-16T15:00:42.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.960+0000 7fd55f400640 1 --2- 172.21.15.143:0/382041297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd560072160 0x7fd56009a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:42.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.960+0000 7fd55fe00640 1 --2- 172.21.15.143:0/382041297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd56006de10 0x7fd5600c60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:42.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.960+0000 7fd564e00640 1 --2- 172.21.15.143:0/382041297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5600c49f0 0x7fd56009c250 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd55800b9e0 tx=0x7fd55800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:42.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.960+0000 7fd557e00640 1 -- 172.21.15.143:0/382041297 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55800cb20 con 0x7fd5600c49f0 2024-01-16T15:00:42.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.961+0000 7fd557e00640 1 -- 172.21.15.143:0/382041297 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd55800cc80 con 0x7fd5600c49f0 2024-01-16T15:00:42.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.961+0000 7fd566e00640 1 -- 172.21.15.143:0/382041297 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd56015be80 con 0x7fd5600c49f0 2024-01-16T15:00:42.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.961+0000 7fd557e00640 1 -- 172.21.15.143:0/382041297 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd558014680 con 0x7fd5600c49f0 2024-01-16T15:00:42.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.961+0000 7fd566e00640 1 -- 172.21.15.143:0/382041297 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd56009ab70 con 0x7fd5600c49f0 2024-01-16T15:00:42.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.962+0000 7fd566e00640 1 -- 172.21.15.143:0/382041297 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd56015c3a0 con 0x7fd5600c49f0 2024-01-16T15:00:42.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.963+0000 7fd557e00640 1 -- 172.21.15.143:0/382041297 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fd55801d3b0 con 0x7fd5600c49f0 2024-01-16T15:00:42.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.964+0000 7fd557e00640 1 --2- 172.21.15.143:0/382041297 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd53806b2c0 0x7fd53806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:42.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.964+0000 7fd557e00640 1 -- 172.21.15.143:0/382041297 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd558089a50 con 0x7fd5600c49f0 2024-01-16T15:00:42.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.964+0000 7fd55fe00640 1 --2- 172.21.15.143:0/382041297 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd53806b2c0 0x7fd53806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:42.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.964+0000 7fd55fe00640 1 --2- 172.21.15.143:0/382041297 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd53806b2c0 0x7fd53806d780 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd550007fd0 tx=0x7fd550009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:42.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:42.968+0000 7fd557e00640 1 -- 172.21.15.143:0/382041297 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd56015c3a0 con 0x7fd5600c49f0 2024-01-16T15:00:43.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.156+0000 7fd566e00640 1 -- 172.21.15.143:0/382041297 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd56015c3a0 con 0x7fd5600c49f0 2024-01-16T15:00:43.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.156+0000 7fd557e00640 1 -- 172.21.15.143:0/382041297 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd56015c3a0 con 0x7fd5600c49f0 2024-01-16T15:00:43.136 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2024-01-16T15:00:43.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.159+0000 7fd566e00640 1 -- 172.21.15.143:0/382041297 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd53806b2c0 msgr2=0x7fd53806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:43.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.159+0000 7fd566e00640 1 --2- 172.21.15.143:0/382041297 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd53806b2c0 0x7fd53806d780 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd550007fd0 tx=0x7fd550009040 comp rx=0 tx=0).stop 2024-01-16T15:00:43.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.159+0000 7fd566e00640 1 -- 172.21.15.143:0/382041297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5600c49f0 msgr2=0x7fd56009c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:43.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.159+0000 7fd566e00640 1 --2- 172.21.15.143:0/382041297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5600c49f0 0x7fd56009c250 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd55800b9e0 tx=0x7fd55800bea0 comp rx=0 tx=0).stop 2024-01-16T15:00:43.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.159+0000 7fd566e00640 1 -- 172.21.15.143:0/382041297 shutdown_connections 2024-01-16T15:00:43.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.159+0000 7fd566e00640 1 --2- 172.21.15.143:0/382041297 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd53806b2c0 0x7fd53806d780 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:43.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.159+0000 7fd566e00640 1 --2- 172.21.15.143:0/382041297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5600c49f0 0x7fd56009c250 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:43.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.159+0000 7fd566e00640 1 --2- 172.21.15.143:0/382041297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd560072160 0x7fd56009a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:43.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.159+0000 7fd566e00640 1 --2- 172.21.15.143:0/382041297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd56006de10 0x7fd5600c60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:43.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.159+0000 7fd566e00640 1 -- 172.21.15.143:0/382041297 >> 172.21.15.143:0/382041297 conn(0x7fd5600b0c50 msgr2=0x7fd5600b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:43.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.160+0000 7fd566e00640 1 -- 172.21.15.143:0/382041297 shutdown_connections 2024-01-16T15:00:43.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.160+0000 7fd566e00640 1 -- 172.21.15.143:0/382041297 wait complete. 2024-01-16T15:00:43.158 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bar_for_1_clone' 2024-01-16T15:00:43.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.376+0000 7f0706600640 1 Processor -- start 2024-01-16T15:00:43.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.377+0000 7f0706600640 1 -- start start 2024-01-16T15:00:43.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.378+0000 7f0706600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0700159ce0 0x7f070006de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:43.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.378+0000 7f0706600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f07000b21d0 0x7f07000cba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:43.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.378+0000 7f0706600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07000b3a40 con 0x7f07000cbf80 2024-01-16T15:00:43.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.378+0000 7f0706600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07000cc410 con 0x7f0700159ce0 2024-01-16T15:00:43.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.378+0000 7f0706600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07000cdfa0 con 0x7f07000b21d0 2024-01-16T15:00:43.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.379+0000 7f06ffe00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f07000cbf80 0x7f070015d760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49314/0 (socket says 172.21.15.143:49314) 2024-01-16T15:00:43.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.379+0000 7f06ffe00640 1 -- 172.21.15.143:0/1092866936 learned_addr learned my addr 172.21.15.143:0/1092866936 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:43.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.379+0000 7f06f6800640 1 --2- 172.21.15.143:0/1092866936 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f07000b21d0 0x7f07000cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:43.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.379+0000 7f06ff400640 1 --2- 172.21.15.143:0/1092866936 >> v2:172.21.15.204:3300/0 conn(0x7f0700159ce0 0x7f070006de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:43.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.379+0000 7f06fea00640 1 -- 172.21.15.143:0/1092866936 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 676693642 0 0) 0x7f07000b3a40 con 0x7f07000cbf80 2024-01-16T15:00:43.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.379+0000 7f06fea00640 1 -- 172.21.15.143:0/1092866936 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06e0003520 con 0x7f07000cbf80 2024-01-16T15:00:43.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.380+0000 7f06fea00640 1 -- 172.21.15.143:0/1092866936 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4023781720 0 0) 0x7f06e0003520 con 0x7f07000cbf80 2024-01-16T15:00:43.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.380+0000 7f06fea00640 1 -- 172.21.15.143:0/1092866936 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f07000b21d0 msgr2=0x7f07000cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:43.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.380+0000 7f06fea00640 1 --2- 172.21.15.143:0/1092866936 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f07000b21d0 0x7f07000cba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:43.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.380+0000 7f06fea00640 1 -- 172.21.15.143:0/1092866936 >> v2:172.21.15.204:3300/0 conn(0x7f0700159ce0 msgr2=0x7f070006de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:43.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.380+0000 7f06fea00640 1 --2- 172.21.15.143:0/1092866936 >> v2:172.21.15.204:3300/0 conn(0x7f0700159ce0 0x7f070006de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:43.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.380+0000 7f06fea00640 1 -- 172.21.15.143:0/1092866936 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07000cd9e0 con 0x7f07000cbf80 2024-01-16T15:00:43.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.380+0000 7f06fea00640 1 -- 172.21.15.143:0/1092866936 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7f06f80019f0 con 0x7f07000cbf80 2024-01-16T15:00:43.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.380+0000 7f06fea00640 1 -- 172.21.15.143:0/1092866936 >> v1:172.21.15.143:6789/0 conn(0x7f07000cbf80 legacy=0x7f070015d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:43.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.380+0000 7f06fea00640 1 --2- 172.21.15.143:0/1092866936 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f07000b21d0 0x7f06e0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:43.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.380+0000 7f06fea00640 1 --2- 172.21.15.143:0/1092866936 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06e0003ef0 0x7f06e00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:43.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.380+0000 7f06fea00640 1 --2- 172.21.15.143:0/1092866936 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06e0004d00 0x7f06e0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:43.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.380+0000 7f06fea00640 1 -- 172.21.15.143:0/1092866936 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07000cc410 con 0x7f06e0004d00 2024-01-16T15:00:43.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.380+0000 7f06fea00640 1 -- 172.21.15.143:0/1092866936 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07000cdfa0 con 0x7f06e0003ef0 2024-01-16T15:00:43.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.380+0000 7f06fea00640 1 -- 172.21.15.143:0/1092866936 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06e000b980 con 0x7f07000b21d0 2024-01-16T15:00:43.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.380+0000 7f06ffe00640 1 --2- 172.21.15.143:0/1092866936 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06e0004d00 0x7f06e0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:43.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.380+0000 7f06ff400640 1 --2- 172.21.15.143:0/1092866936 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f07000b21d0 0x7f06e0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:43.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.380+0000 7f06f6800640 1 --2- 172.21.15.143:0/1092866936 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06e0003ef0 0x7f06e00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:43.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.381+0000 7f06ffe00640 1 -- 172.21.15.143:0/1092866936 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f07000b21d0 msgr2=0x7f06e0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:43.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.381+0000 7f06ffe00640 1 --2- 172.21.15.143:0/1092866936 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f07000b21d0 0x7f06e0003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:43.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.381+0000 7f06ffe00640 1 -- 172.21.15.143:0/1092866936 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06e0003ef0 msgr2=0x7f06e00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:43.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.381+0000 7f06ffe00640 1 --2- 172.21.15.143:0/1092866936 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06e0003ef0 0x7f06e00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:43.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.381+0000 7f06ffe00640 1 -- 172.21.15.143:0/1092866936 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e0003520 con 0x7f06e0004d00 2024-01-16T15:00:43.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.382+0000 7f06ffe00640 1 --2- 172.21.15.143:0/1092866936 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06e0004d00 0x7f06e0005180 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f06f800a860 tx=0x7f06e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1663e0b914ddf918 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:43.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.382+0000 7f06fea00640 1 -- 172.21.15.143:0/1092866936 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f8034630 con 0x7f06e0004d00 2024-01-16T15:00:43.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.382+0000 7f06fea00640 1 -- 172.21.15.143:0/1092866936 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f802f070 con 0x7f06e0004d00 2024-01-16T15:00:43.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.382+0000 7f06fea00640 1 -- 172.21.15.143:0/1092866936 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f8034a30 con 0x7f06e0004d00 2024-01-16T15:00:43.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.383+0000 7f0706600640 1 -- 172.21.15.143:0/1092866936 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06e0004d00 msgr2=0x7f06e0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:43.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.383+0000 7f0706600640 1 --2- 172.21.15.143:0/1092866936 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06e0004d00 0x7f06e0005180 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f06f800a860 tx=0x7f06e00057f0 comp rx=0 tx=0).stop 2024-01-16T15:00:43.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.383+0000 7f06ffe00640 1 -- 172.21.15.143:0/1092866936 reap_dead start 2024-01-16T15:00:43.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.383+0000 7f0706600640 1 -- 172.21.15.143:0/1092866936 shutdown_connections 2024-01-16T15:00:43.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.383+0000 7f0706600640 1 -- 172.21.15.143:0/1092866936 >> 172.21.15.143:0/1092866936 conn(0x7f070006a920 msgr2=0x7f070006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:43.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.383+0000 7f0706600640 1 -- 172.21.15.143:0/1092866936 shutdown_connections 2024-01-16T15:00:43.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.383+0000 7f0706600640 1 -- 172.21.15.143:0/1092866936 wait complete. 2024-01-16T15:00:43.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.385+0000 7f0706600640 1 Processor -- start 2024-01-16T15:00:43.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.385+0000 7f0706600640 1 -- start start 2024-01-16T15:00:43.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.386+0000 7f0706600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06e0004d00 0x7f07000c8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:43.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.386+0000 7f0706600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f07000c8cb0 0x7f0700071d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:43.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.386+0000 7f0706600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0700098130 0x7f0700072270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:43.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.386+0000 7f0706600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07000cacf0 con 0x7f07000c8cb0 2024-01-16T15:00:43.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.386+0000 7f0706600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07000ca400 con 0x7f0700098130 2024-01-16T15:00:43.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.386+0000 7f0706600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07000988b0 con 0x7f06e0004d00 2024-01-16T15:00:43.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.387+0000 7f06f6800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f07000c8cb0 0x7f0700071d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:43.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.387+0000 7f06f6800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f07000c8cb0 0x7f0700071d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43426/0 (socket says 172.21.15.143:43426) 2024-01-16T15:00:43.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.387+0000 7f06f6800640 1 -- 172.21.15.143:0/1533527195 learned_addr learned my addr 172.21.15.143:0/1533527195 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:43.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.387+0000 7f06ff400640 1 --2- 172.21.15.143:0/1533527195 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06e0004d00 0x7f07000c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:43.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.387+0000 7f06f6800640 1 -- 172.21.15.143:0/1533527195 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06e0004d00 msgr2=0x7f07000c8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:43.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.387+0000 7f06f6800640 1 --2- 172.21.15.143:0/1533527195 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06e0004d00 0x7f07000c8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:43.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.387+0000 7f06f6800640 1 -- 172.21.15.143:0/1533527195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0700098130 msgr2=0x7f0700072270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:43.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.387+0000 7f06f6800640 1 --2- 172.21.15.143:0/1533527195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0700098130 0x7f0700072270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:43.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.387+0000 7f06f6800640 1 -- 172.21.15.143:0/1533527195 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0700145180 con 0x7f07000c8cb0 2024-01-16T15:00:43.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.388+0000 7f06f6800640 1 --2- 172.21.15.143:0/1533527195 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f07000c8cb0 0x7f0700071d30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f06e8011540 tx=0x7f06e8011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:43.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.388+0000 7f06f7e00640 1 -- 172.21.15.143:0/1533527195 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e8015930 con 0x7f07000c8cb0 2024-01-16T15:00:43.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.388+0000 7f06f7e00640 1 -- 172.21.15.143:0/1533527195 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06e8015a90 con 0x7f07000c8cb0 2024-01-16T15:00:43.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.388+0000 7f06f7e00640 1 -- 172.21.15.143:0/1533527195 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e8015e60 con 0x7f07000c8cb0 2024-01-16T15:00:43.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.388+0000 7f0706600640 1 -- 172.21.15.143:0/1533527195 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0700144ea0 con 0x7f07000c8cb0 2024-01-16T15:00:43.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.388+0000 7f0706600640 1 -- 172.21.15.143:0/1533527195 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07000ca910 con 0x7f07000c8cb0 2024-01-16T15:00:43.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.389+0000 7f0706600640 1 -- 172.21.15.143:0/1533527195 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07000cae30 con 0x7f07000c8cb0 2024-01-16T15:00:43.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.390+0000 7f06f7e00640 1 -- 172.21.15.143:0/1533527195 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f06e801d400 con 0x7f07000c8cb0 2024-01-16T15:00:43.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.391+0000 7f06f7e00640 1 --2- 172.21.15.143:0/1533527195 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f06d806b2f0 0x7f06d806d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:43.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.392+0000 7f06f7e00640 1 -- 172.21.15.143:0/1533527195 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f06e808a3a0 con 0x7f07000c8cb0 2024-01-16T15:00:43.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.392+0000 7f06ff400640 1 --2- 172.21.15.143:0/1533527195 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f06d806b2f0 0x7f06d806d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:43.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.392+0000 7f06ff400640 1 --2- 172.21.15.143:0/1533527195 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f06d806b2f0 0x7f06d806d7b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f06f00099d0 tx=0x7f06f0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:43.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.396+0000 7f06f7e00640 1 -- 172.21.15.143:0/1533527195 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f07000cae30 con 0x7f07000c8cb0 2024-01-16T15:00:43.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:43.662+0000 7f0706600640 1 -- 172.21.15.143:0/1533527195 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bar_for_1_clone"]} v 0) v1 -- 0x7f07000cae30 con 0x7f07000c8cb0 2024-01-16T15:00:44.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.626+0000 7f06f7e00640 1 -- 172.21.15.143:0/1533527195 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bar_for_1_clone"]}]=0 v60) v1 ==== 155+0+0 (secure 0 0 0) 0x7f07000cae30 con 0x7f07000c8cb0 2024-01-16T15:00:44.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.631+0000 7f0706600640 1 -- 172.21.15.143:0/1533527195 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f06d806b2f0 msgr2=0x7f06d806d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:44.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.631+0000 7f0706600640 1 --2- 172.21.15.143:0/1533527195 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f06d806b2f0 0x7f06d806d7b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f06f00099d0 tx=0x7f06f0008040 comp rx=0 tx=0).stop 2024-01-16T15:00:44.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.631+0000 7f0706600640 1 -- 172.21.15.143:0/1533527195 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f07000c8cb0 msgr2=0x7f0700071d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:44.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.631+0000 7f0706600640 1 --2- 172.21.15.143:0/1533527195 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f07000c8cb0 0x7f0700071d30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f06e8011540 tx=0x7f06e8011a00 comp rx=0 tx=0).stop 2024-01-16T15:00:44.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.632+0000 7f0706600640 1 -- 172.21.15.143:0/1533527195 shutdown_connections 2024-01-16T15:00:44.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.632+0000 7f0706600640 1 --2- 172.21.15.143:0/1533527195 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f06d806b2f0 0x7f06d806d7b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:44.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.632+0000 7f0706600640 1 --2- 172.21.15.143:0/1533527195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0700098130 0x7f0700072270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:44.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.632+0000 7f0706600640 1 --2- 172.21.15.143:0/1533527195 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f07000c8cb0 0x7f0700071d30 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:44.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.632+0000 7f0706600640 1 --2- 172.21.15.143:0/1533527195 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06e0004d00 0x7f07000c8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:44.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.632+0000 7f0706600640 1 -- 172.21.15.143:0/1533527195 >> 172.21.15.143:0/1533527195 conn(0x7f070006a920 msgr2=0x7f07000705c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:44.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.632+0000 7f0706600640 1 -- 172.21.15.143:0/1533527195 shutdown_connections 2024-01-16T15:00:44.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.632+0000 7f0706600640 1 -- 172.21.15.143:0/1533527195 wait complete. 2024-01-16T15:00:44.634 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-16T15:00:44.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.850+0000 7f1bb8000640 1 Processor -- start 2024-01-16T15:00:44.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.851+0000 7f1bb8000640 1 -- start start 2024-01-16T15:00:44.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.852+0000 7f1bb8000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1bb00cc0f0 0x7f1bb00c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:44.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.852+0000 7f1bb8000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bb015eea0 con 0x7f1bb00c57e0 2024-01-16T15:00:44.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.852+0000 7f1bb8000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bb00abf30 con 0x7f1bb015b440 2024-01-16T15:00:44.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.852+0000 7f1bb8000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bb015b820 con 0x7f1bb00cc0f0 2024-01-16T15:00:44.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.853+0000 7f1bb6000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f1bb00c57e0 0x7f1bb006c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49328/0 (socket says 172.21.15.143:49328) 2024-01-16T15:00:44.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.853+0000 7f1bb6000640 1 -- 172.21.15.143:0/2457340605 learned_addr learned my addr 172.21.15.143:0/2457340605 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:44.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.853+0000 7f1bafe00640 1 -- 172.21.15.143:0/2457340605 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 660374226 0 0) 0x7f1bb015eea0 con 0x7f1bb00c57e0 2024-01-16T15:00:44.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.853+0000 7f1bafe00640 1 -- 172.21.15.143:0/2457340605 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b90003520 con 0x7f1bb00c57e0 2024-01-16T15:00:44.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.853+0000 7f1bafe00640 1 -- 172.21.15.143:0/2457340605 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 17164154 0 0) 0x7f1b90003520 con 0x7f1bb00c57e0 2024-01-16T15:00:44.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.853+0000 7f1bafe00640 1 -- 172.21.15.143:0/2457340605 >> v2:172.21.15.204:3300/0 conn(0x7f1bb00cc0f0 msgr2=0x7f1bb00c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:44.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.853+0000 7f1bafe00640 1 --2- 172.21.15.143:0/2457340605 >> v2:172.21.15.204:3300/0 conn(0x7f1bb00cc0f0 0x7f1bb00c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:44.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.853+0000 7f1bafe00640 1 -- 172.21.15.143:0/2457340605 >> v1:172.21.15.204:6789/0 conn(0x7f1bb015b440 legacy=0x7f1bb00cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:44.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.853+0000 7f1bafe00640 1 -- 172.21.15.143:0/2457340605 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb015f0f0 con 0x7f1bb00c57e0 2024-01-16T15:00:44.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.853+0000 7f1bafe00640 1 -- 172.21.15.143:0/2457340605 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7f1ba8001670 con 0x7f1bb00c57e0 2024-01-16T15:00:44.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.853+0000 7f1bafe00640 1 -- 172.21.15.143:0/2457340605 >> v1:172.21.15.143:6789/0 conn(0x7f1bb00c57e0 legacy=0x7f1bb006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:44.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.853+0000 7f1bafe00640 1 --2- 172.21.15.143:0/2457340605 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b90003970 0x7f1b90003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:44.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.853+0000 7f1bafe00640 1 --2- 172.21.15.143:0/2457340605 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b900047c0 0x7f1b90004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:44.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.853+0000 7f1bafe00640 1 --2- 172.21.15.143:0/2457340605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b90005380 0x7f1b9000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:44.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.853+0000 7f1bafe00640 1 -- 172.21.15.143:0/2457340605 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb015b820 con 0x7f1b90003970 2024-01-16T15:00:44.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.853+0000 7f1bafe00640 1 -- 172.21.15.143:0/2457340605 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9000c220 con 0x7f1b90005380 2024-01-16T15:00:44.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.853+0000 7f1bafe00640 1 -- 172.21.15.143:0/2457340605 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9000c3e0 con 0x7f1b900047c0 2024-01-16T15:00:44.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.854+0000 7f1bb5600640 1 --2- 172.21.15.143:0/2457340605 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b90003970 0x7f1b90003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:44.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.854+0000 7f1bb6000640 1 --2- 172.21.15.143:0/2457340605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b90005380 0x7f1b9000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:44.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.854+0000 7f1bb4c00640 1 --2- 172.21.15.143:0/2457340605 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b900047c0 0x7f1b90004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:44.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.854+0000 7f1bb5600640 1 -- 172.21.15.143:0/2457340605 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b900047c0 msgr2=0x7f1b90004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:44.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.854+0000 7f1bb5600640 1 --2- 172.21.15.143:0/2457340605 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b900047c0 0x7f1b90004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:44.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.854+0000 7f1bb5600640 1 -- 172.21.15.143:0/2457340605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b90005380 msgr2=0x7f1b9000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:44.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.854+0000 7f1bb5600640 1 --2- 172.21.15.143:0/2457340605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b90005380 0x7f1b9000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:44.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.854+0000 7f1bb5600640 1 -- 172.21.15.143:0/2457340605 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b90003520 con 0x7f1b90003970 2024-01-16T15:00:44.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.854+0000 7f1bb6000640 1 --2- 172.21.15.143:0/2457340605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b90005380 0x7f1b9000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:44.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.854+0000 7f1bb6000640 1 -- 172.21.15.143:0/2457340605 reap_dead start 2024-01-16T15:00:44.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.855+0000 7f1bb5600640 1 --2- 172.21.15.143:0/2457340605 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b90003970 0x7f1b90003d70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1ba00044b0 tx=0x7f1b900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14217ac47380609a server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:44.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.856+0000 7f1bafe00640 1 -- 172.21.15.143:0/2457340605 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0030400 con 0x7f1b90003970 2024-01-16T15:00:44.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.856+0000 7f1bafe00640 1 -- 172.21.15.143:0/2457340605 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba0030560 con 0x7f1b90003970 2024-01-16T15:00:44.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.856+0000 7f1bafe00640 1 -- 172.21.15.143:0/2457340605 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0030950 con 0x7f1b90003970 2024-01-16T15:00:44.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.856+0000 7f1bb8000640 1 -- 172.21.15.143:0/2457340605 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b90003970 msgr2=0x7f1b90003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:44.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.856+0000 7f1bb8000640 1 --2- 172.21.15.143:0/2457340605 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b90003970 0x7f1b90003d70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1ba00044b0 tx=0x7f1b900057f0 comp rx=0 tx=0).stop 2024-01-16T15:00:44.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.856+0000 7f1bb8000640 1 -- 172.21.15.143:0/2457340605 shutdown_connections 2024-01-16T15:00:44.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.856+0000 7f1bb8000640 1 --2- 172.21.15.143:0/2457340605 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b90003970 0x7f1b90003d70 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:44.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.856+0000 7f1bb8000640 1 -- 172.21.15.143:0/2457340605 >> 172.21.15.143:0/2457340605 conn(0x7f1bb00b0c50 msgr2=0x7f1bb00b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:44.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.856+0000 7f1bb8000640 1 -- 172.21.15.143:0/2457340605 shutdown_connections 2024-01-16T15:00:44.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.856+0000 7f1bb8000640 1 -- 172.21.15.143:0/2457340605 wait complete. 2024-01-16T15:00:44.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.858+0000 7f1bb8000640 1 Processor -- start 2024-01-16T15:00:44.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.858+0000 7f1bb8000640 1 -- start start 2024-01-16T15:00:44.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.859+0000 7f1bb8000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b90003970 0x7f1bb00cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:44.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.859+0000 7f1bb8000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1bb0156cb0 0x7f1bb01f6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:44.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.859+0000 7f1bb8000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bb0157110 0x7f1bb01f6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:44.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.859+0000 7f1bb8000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb00ce540 con 0x7f1bb0157110 2024-01-16T15:00:44.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.859+0000 7f1bb8000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb00cdd60 con 0x7f1bb0156cb0 2024-01-16T15:00:44.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.859+0000 7f1bb8000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb0154150 con 0x7f1b90003970 2024-01-16T15:00:44.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.859+0000 7f1bb6000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bb0157110 0x7f1bb01f6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:44.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.860+0000 7f1bb6000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bb0157110 0x7f1bb01f6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43442/0 (socket says 172.21.15.143:43442) 2024-01-16T15:00:44.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.860+0000 7f1bb6000640 1 -- 172.21.15.143:0/102772610 learned_addr learned my addr 172.21.15.143:0/102772610 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:44.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.860+0000 7f1bb5600640 1 --2- 172.21.15.143:0/102772610 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b90003970 0x7f1bb00cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:44.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.860+0000 7f1bb4c00640 1 --2- 172.21.15.143:0/102772610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1bb0156cb0 0x7f1bb01f6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:44.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.860+0000 7f1bb6000640 1 -- 172.21.15.143:0/102772610 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b90003970 msgr2=0x7f1bb00cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:44.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.860+0000 7f1bb6000640 1 --2- 172.21.15.143:0/102772610 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b90003970 0x7f1bb00cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:44.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.860+0000 7f1bb6000640 1 -- 172.21.15.143:0/102772610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1bb0156cb0 msgr2=0x7f1bb01f6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:44.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.860+0000 7f1bb6000640 1 --2- 172.21.15.143:0/102772610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1bb0156cb0 0x7f1bb01f6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:44.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.860+0000 7f1bb6000640 1 -- 172.21.15.143:0/102772610 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb0154430 con 0x7f1bb0157110 2024-01-16T15:00:44.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.860+0000 7f1bb6000640 1 --2- 172.21.15.143:0/102772610 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bb0157110 0x7f1bb01f6a60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1bb009d280 tx=0x7f1ba800d900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:44.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.861+0000 7f1bad600640 1 -- 172.21.15.143:0/102772610 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba8007440 con 0x7f1bb0157110 2024-01-16T15:00:44.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.861+0000 7f1bad600640 1 -- 172.21.15.143:0/102772610 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba80075a0 con 0x7f1bb0157110 2024-01-16T15:00:44.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.861+0000 7f1bad600640 1 -- 172.21.15.143:0/102772610 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba8004a20 con 0x7f1bb0157110 2024-01-16T15:00:44.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.861+0000 7f1bb8000640 1 -- 172.21.15.143:0/102772610 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bb0154710 con 0x7f1bb0157110 2024-01-16T15:00:44.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.861+0000 7f1bb8000640 1 -- 172.21.15.143:0/102772610 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bb01f6fa0 con 0x7f1bb0157110 2024-01-16T15:00:44.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.862+0000 7f1bb8000640 1 -- 172.21.15.143:0/102772610 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bb00c6fe0 con 0x7f1bb0157110 2024-01-16T15:00:44.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.863+0000 7f1bad600640 1 -- 172.21.15.143:0/102772610 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f1ba8004b80 con 0x7f1bb0157110 2024-01-16T15:00:44.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.864+0000 7f1bad600640 1 --2- 172.21.15.143:0/102772610 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f1b8806b2c0 0x7f1b8806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:44.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.864+0000 7f1bad600640 1 -- 172.21.15.143:0/102772610 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f1ba80895a0 con 0x7f1bb0157110 2024-01-16T15:00:44.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.864+0000 7f1bb5600640 1 --2- 172.21.15.143:0/102772610 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f1b8806b2c0 0x7f1b8806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:44.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.865+0000 7f1bb5600640 1 --2- 172.21.15.143:0/102772610 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f1b8806b2c0 0x7f1b8806d780 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0036000 tx=0x7f1ba003c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:44.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:44.869+0000 7f1bad600640 1 -- 172.21.15.143:0/102772610 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1ba8054430 con 0x7f1bb0157110 2024-01-16T15:00:45.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.058+0000 7f1bb8000640 1 -- 172.21.15.143:0/102772610 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f1bb004e8b0 con 0x7f1bb0157110 2024-01-16T15:00:45.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.065+0000 7f1bad600640 1 -- 172.21.15.143:0/102772610 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1ba8057a50 con 0x7f1bb0157110 2024-01-16T15:00:45.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.067+0000 7f1bb8000640 1 -- 172.21.15.143:0/102772610 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f1b8806b2c0 msgr2=0x7f1b8806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:45.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.067+0000 7f1bb8000640 1 --2- 172.21.15.143:0/102772610 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f1b8806b2c0 0x7f1b8806d780 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0036000 tx=0x7f1ba003c040 comp rx=0 tx=0).stop 2024-01-16T15:00:45.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.067+0000 7f1bb8000640 1 -- 172.21.15.143:0/102772610 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bb0157110 msgr2=0x7f1bb01f6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:45.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.067+0000 7f1bb8000640 1 --2- 172.21.15.143:0/102772610 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bb0157110 0x7f1bb01f6a60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1bb009d280 tx=0x7f1ba800d900 comp rx=0 tx=0).stop 2024-01-16T15:00:45.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.068+0000 7f1bb8000640 1 -- 172.21.15.143:0/102772610 shutdown_connections 2024-01-16T15:00:45.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.068+0000 7f1bb8000640 1 --2- 172.21.15.143:0/102772610 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f1b8806b2c0 0x7f1b8806d780 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:45.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.068+0000 7f1bb8000640 1 --2- 172.21.15.143:0/102772610 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bb0157110 0x7f1bb01f6a60 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:45.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.068+0000 7f1bb8000640 1 --2- 172.21.15.143:0/102772610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1bb0156cb0 0x7f1bb01f6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:45.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.068+0000 7f1bb8000640 1 --2- 172.21.15.143:0/102772610 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b90003970 0x7f1bb00cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:45.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.068+0000 7f1bb8000640 1 -- 172.21.15.143:0/102772610 >> 172.21.15.143:0/102772610 conn(0x7f1bb00b0c50 msgr2=0x7f1bb00cb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:45.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.068+0000 7f1bb8000640 1 -- 172.21.15.143:0/102772610 shutdown_connections 2024-01-16T15:00:45.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.068+0000 7f1bb8000640 1 -- 172.21.15.143:0/102772610 wait complete. 2024-01-16T15:00:45.066 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-16T15:00:45.066 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-16T15:00:45.179 DEBUG:teuthology.orchestra.run.smithi204:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-16T15:00:45.304 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-16T15:00:45.304 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T15:00:45.304 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T15:00:45.361 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T15:00:45.361 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T15:00:45.413 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T15:00:45.414 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T15:00:45.465 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T15:00:45.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.699+0000 7f5987e00640 1 Processor -- start 2024-01-16T15:00:45.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.699+0000 7f5987e00640 1 -- start start 2024-01-16T15:00:45.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.701+0000 7f5987e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5988143870 0x7f5988146970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:45.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.701+0000 7f5987e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f59880b1180 0x7f59880cd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:45.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.701+0000 7f5987e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5988161510 con 0x7f5988161080 2024-01-16T15:00:45.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.701+0000 7f5987e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5988070460 con 0x7f59880b1180 2024-01-16T15:00:45.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.701+0000 7f5987e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59881617f0 con 0x7f5988143870 2024-01-16T15:00:45.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.701+0000 7f597de00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f5988161080 0x7f59880b0a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50528/0 (socket says 172.21.15.143:50528) 2024-01-16T15:00:45.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.701+0000 7f597de00640 1 -- 172.21.15.143:0/412926407 learned_addr learned my addr 172.21.15.143:0/412926407 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:45.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.702+0000 7f5987400640 1 --2- 172.21.15.143:0/412926407 >> v2:172.21.15.204:3300/0 conn(0x7f59880b1180 0x7f59880cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:45.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.702+0000 7f5986a00640 1 --2- 172.21.15.143:0/412926407 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5988143870 0x7f5988146970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:45.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.702+0000 7f5986000640 1 -- 172.21.15.143:0/412926407 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3973970528 0 0) 0x7f5988161510 con 0x7f5988161080 2024-01-16T15:00:45.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.702+0000 7f5986000640 1 -- 172.21.15.143:0/412926407 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5968003520 con 0x7f5988161080 2024-01-16T15:00:45.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.702+0000 7f5986000640 1 -- 172.21.15.143:0/412926407 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 788261374 0 0) 0x7f5968003520 con 0x7f5988161080 2024-01-16T15:00:45.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.702+0000 7f5986000640 1 -- 172.21.15.143:0/412926407 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5988143870 msgr2=0x7f5988146970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:45.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.702+0000 7f5986000640 1 --2- 172.21.15.143:0/412926407 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5988143870 0x7f5988146970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:45.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.702+0000 7f5986000640 1 -- 172.21.15.143:0/412926407 >> v2:172.21.15.204:3300/0 conn(0x7f59880b1180 msgr2=0x7f59880cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:45.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.702+0000 7f5986000640 1 --2- 172.21.15.143:0/412926407 >> v2:172.21.15.204:3300/0 conn(0x7f59880b1180 0x7f59880cd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:45.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.702+0000 7f5986000640 1 -- 172.21.15.143:0/412926407 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5988161ad0 con 0x7f5988161080 2024-01-16T15:00:45.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.702+0000 7f5986000640 1 -- 172.21.15.143:0/412926407 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7f5970001670 con 0x7f5988161080 2024-01-16T15:00:45.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.702+0000 7f5986000640 1 -- 172.21.15.143:0/412926407 >> v1:172.21.15.143:6789/0 conn(0x7f5988161080 legacy=0x7f59880b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:45.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.702+0000 7f5986000640 1 --2- 172.21.15.143:0/412926407 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5988143870 0x7f5968003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:45.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.702+0000 7f5986000640 1 --2- 172.21.15.143:0/412926407 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5968003ef0 0x7f59680047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:45.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.702+0000 7f5986000640 1 --2- 172.21.15.143:0/412926407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5968004d00 0x7f5968005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:45.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.702+0000 7f5986000640 1 -- 172.21.15.143:0/412926407 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5988070460 con 0x7f5968003ef0 2024-01-16T15:00:45.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.702+0000 7f5986000640 1 -- 172.21.15.143:0/412926407 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59881617f0 con 0x7f5968004d00 2024-01-16T15:00:45.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.702+0000 7f5986000640 1 -- 172.21.15.143:0/412926407 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f596800b980 con 0x7f5988143870 2024-01-16T15:00:45.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.703+0000 7f597de00640 1 --2- 172.21.15.143:0/412926407 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5968003ef0 0x7f59680047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:45.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.703+0000 7f5987400640 1 --2- 172.21.15.143:0/412926407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5968004d00 0x7f5968005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:45.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.703+0000 7f5986a00640 1 --2- 172.21.15.143:0/412926407 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5988143870 0x7f5968003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:45.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.703+0000 7f597de00640 1 -- 172.21.15.143:0/412926407 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5988143870 msgr2=0x7f5968003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:45.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.703+0000 7f597de00640 1 --2- 172.21.15.143:0/412926407 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5988143870 0x7f5968003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:45.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.703+0000 7f597de00640 1 -- 172.21.15.143:0/412926407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5968004d00 msgr2=0x7f5968005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:45.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.703+0000 7f597de00640 1 --2- 172.21.15.143:0/412926407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5968004d00 0x7f5968005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:45.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.703+0000 7f597de00640 1 -- 172.21.15.143:0/412926407 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5968003520 con 0x7f5968003ef0 2024-01-16T15:00:45.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.703+0000 7f5987400640 1 --2- 172.21.15.143:0/412926407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5968004d00 0x7f5968005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:45.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.704+0000 7f597de00640 1 --2- 172.21.15.143:0/412926407 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5968003ef0 0x7f59680047c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f597000c240 tx=0x7f59680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef0397c2edb66eac server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:45.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.704+0000 7f5986000640 1 -- 172.21.15.143:0/412926407 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597002cae0 con 0x7f5968003ef0 2024-01-16T15:00:45.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.704+0000 7f5986000640 1 -- 172.21.15.143:0/412926407 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f597002cc40 con 0x7f5968003ef0 2024-01-16T15:00:45.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.704+0000 7f5986000640 1 -- 172.21.15.143:0/412926407 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5970005af0 con 0x7f5968003ef0 2024-01-16T15:00:45.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.705+0000 7f5987e00640 1 -- 172.21.15.143:0/412926407 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5968003ef0 msgr2=0x7f59680047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:45.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.705+0000 7f5987e00640 1 --2- 172.21.15.143:0/412926407 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5968003ef0 0x7f59680047c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f597000c240 tx=0x7f59680057f0 comp rx=0 tx=0).stop 2024-01-16T15:00:45.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.705+0000 7f5987400640 1 -- 172.21.15.143:0/412926407 reap_dead start 2024-01-16T15:00:45.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.705+0000 7f5987e00640 1 -- 172.21.15.143:0/412926407 shutdown_connections 2024-01-16T15:00:45.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.705+0000 7f5987e00640 1 -- 172.21.15.143:0/412926407 >> 172.21.15.143:0/412926407 conn(0x7f59880cb830 msgr2=0x7f59880cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:45.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.705+0000 7f5987e00640 1 -- 172.21.15.143:0/412926407 shutdown_connections 2024-01-16T15:00:45.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.705+0000 7f5987e00640 1 -- 172.21.15.143:0/412926407 wait complete. 2024-01-16T15:00:45.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.707+0000 7f5987e00640 1 Processor -- start 2024-01-16T15:00:45.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.707+0000 7f5987e00640 1 -- start start 2024-01-16T15:00:45.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.708+0000 7f5987e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5968003ef0 0x7f598813e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:45.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.708+0000 7f5987e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59880b2c30 0x7f5988145660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:45.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.708+0000 7f5987e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5988145ba0 0x7f59880711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:45.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.708+0000 7f5987e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59881f6270 con 0x7f59880b2c30 2024-01-16T15:00:45.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.708+0000 7f5987e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59881f7060 con 0x7f5988145ba0 2024-01-16T15:00:45.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.708+0000 7f5987e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59880b30b0 con 0x7f5968003ef0 2024-01-16T15:00:45.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.708+0000 7f597de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59880b2c30 0x7f5988145660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:45.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.708+0000 7f597de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59880b2c30 0x7f5988145660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59736/0 (socket says 172.21.15.143:59736) 2024-01-16T15:00:45.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.708+0000 7f597de00640 1 -- 172.21.15.143:0/532617193 learned_addr learned my addr 172.21.15.143:0/532617193 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:45.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.709+0000 7f5986a00640 1 --2- 172.21.15.143:0/532617193 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5968003ef0 0x7f598813e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:45.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.709+0000 7f5987400640 1 --2- 172.21.15.143:0/532617193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5988145ba0 0x7f59880711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:45.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.709+0000 7f597de00640 1 -- 172.21.15.143:0/532617193 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5968003ef0 msgr2=0x7f598813e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:45.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.709+0000 7f597de00640 1 --2- 172.21.15.143:0/532617193 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5968003ef0 0x7f598813e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:45.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.709+0000 7f597de00640 1 -- 172.21.15.143:0/532617193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5988145ba0 msgr2=0x7f59880711d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:45.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.709+0000 7f597de00640 1 --2- 172.21.15.143:0/532617193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5988145ba0 0x7f59880711d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:45.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.709+0000 7f597de00640 1 -- 172.21.15.143:0/532617193 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59880b4d90 con 0x7f59880b2c30 2024-01-16T15:00:45.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.709+0000 7f597de00640 1 --2- 172.21.15.143:0/532617193 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59880b2c30 0x7f5988145660 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f597002c4e0 tx=0x7f5970001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:45.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.709+0000 7f597f400640 1 -- 172.21.15.143:0/532617193 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5970039030 con 0x7f59880b2c30 2024-01-16T15:00:45.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.710+0000 7f597f400640 1 -- 172.21.15.143:0/532617193 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5970004bf0 con 0x7f59880b2c30 2024-01-16T15:00:45.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.710+0000 7f597f400640 1 -- 172.21.15.143:0/532617193 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597003c040 con 0x7f59880b2c30 2024-01-16T15:00:45.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.710+0000 7f5987e00640 1 -- 172.21.15.143:0/532617193 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59881f6450 con 0x7f59880b2c30 2024-01-16T15:00:45.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.710+0000 7f5987e00640 1 -- 172.21.15.143:0/532617193 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59881f6a10 con 0x7f59880b2c30 2024-01-16T15:00:45.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.711+0000 7f5987e00640 1 -- 172.21.15.143:0/532617193 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59881f71a0 con 0x7f59880b2c30 2024-01-16T15:00:45.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.712+0000 7f597f400640 1 -- 172.21.15.143:0/532617193 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f5970032070 con 0x7f59880b2c30 2024-01-16T15:00:45.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.713+0000 7f597f400640 1 --2- 172.21.15.143:0/532617193 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f596006b2c0 0x7f596006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:45.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.713+0000 7f597f400640 1 -- 172.21.15.143:0/532617193 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f59700ab1b0 con 0x7f59880b2c30 2024-01-16T15:00:45.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.713+0000 7f5986a00640 1 --2- 172.21.15.143:0/532617193 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f596006b2c0 0x7f596006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:45.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.713+0000 7f5986a00640 1 --2- 172.21.15.143:0/532617193 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f596006b2c0 0x7f596006d780 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5978009fd0 tx=0x7f597800d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:45.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.717+0000 7f597f400640 1 -- 172.21.15.143:0/532617193 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f59881f71a0 con 0x7f59880b2c30 2024-01-16T15:00:45.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.910+0000 7f5987e00640 1 -- 172.21.15.143:0/532617193 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59881f71a0 con 0x7f59880b2c30 2024-01-16T15:00:45.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.912+0000 7f597f400640 1 -- 172.21.15.143:0/532617193 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16814 (secure 0 0 0) 0x7f59881f71a0 con 0x7f59880b2c30 2024-01-16T15:00:45.893 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:00:45.893 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":22,"fsid":"c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0","created":"2024-01-16T14:59:38.532601+0000","modified":"2024-01-16T15:00:45.626415+0000","last_up_change":"2024-01-16T14:59:50.983699+0000","last_in_change":"2024-01-16T14:59:42.056297+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:59:51.727720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T15:00:01.010472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T15:00:02.041228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f4c8891d-ab29-472a-907d-8446f20f7012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6801","nonce":1569293626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6805","nonce":1569293626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6814","nonce":1569293626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6809","nonce":1569293626}]},"public_addr":"172.21.15.143:6801/1569293626","cluster_addr":"172.21.15.143:6805/1569293626","heartbeat_back_addr":"172.21.15.143:6814/1569293626","heartbeat_front_addr":"172.21.15.143:6809/1569293626","state":["exists","up"]},{"osd":1,"uuid":"c4a2788d-8b89-4992-82b2-7f4b9debd509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6804","nonce":1276841168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6808","nonce":1276841168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6813","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6815","nonce":1276841168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6811","nonce":1276841168}]},"public_addr":"172.21.15.143:6804/1276841168","cluster_addr":"172.21.15.143:6808/1276841168","heartbeat_back_addr":"172.21.15.143:6815/1276841168","heartbeat_front_addr":"172.21.15.143:6811/1276841168","state":["exists","up"]},{"osd":2,"uuid":"64645c1d-31b2-4794-b0f2-7ccf8df30393","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6817","nonce":1217670170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6819","nonce":1217670170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6823","nonce":1217670170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6821","nonce":1217670170}]},"public_addr":"172.21.15.143:6817/1217670170","cluster_addr":"172.21.15.143:6819/1217670170","heartbeat_back_addr":"172.21.15.143:6823/1217670170","heartbeat_front_addr":"172.21.15.143:6821/1217670170","state":["exists","up"]},{"osd":3,"uuid":"73674129-a84e-4d31-8247-7d0179ae6df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6825","nonce":3414188012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6827","nonce":3414188012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6831","nonce":3414188012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6829","nonce":3414188012}]},"public_addr":"172.21.15.143:6825/3414188012","cluster_addr":"172.21.15.143:6827/3414188012","heartbeat_back_addr":"172.21.15.143:6831/3414188012","heartbeat_front_addr":"172.21.15.143:6829/3414188012","state":["exists","up"]},{"osd":4,"uuid":"cea28c9b-2f1c-4087-b415-9960789131d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6811","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6812","nonce":1070255716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6816","nonce":1070255716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6823","nonce":1070255716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6820","nonce":1070255716}]},"public_addr":"172.21.15.204:6812/1070255716","cluster_addr":"172.21.15.204:6816/1070255716","heartbeat_back_addr":"172.21.15.204:6823/1070255716","heartbeat_front_addr":"172.21.15.204:6820/1070255716","state":["exists","up"]},{"osd":5,"uuid":"33b63560-5574-4e78-983f-b398d67a2cb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6809","nonce":3605417837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6813","nonce":3605417837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6819","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6821","nonce":3605417837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6817","nonce":3605417837}]},"public_addr":"172.21.15.204:6809/3605417837","cluster_addr":"172.21.15.204:6813/3605417837","heartbeat_back_addr":"172.21.15.204:6821/3605417837","heartbeat_front_addr":"172.21.15.204:6817/3605417837","state":["exists","up"]},{"osd":6,"uuid":"df397427-8940-4eee-9c79-eece669d735d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6801","nonce":1085000534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6803","nonce":1085000534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6808","nonce":1085000534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6805","nonce":1085000534}]},"public_addr":"172.21.15.204:6801/1085000534","cluster_addr":"172.21.15.204:6803/1085000534","heartbeat_back_addr":"172.21.15.204:6808/1085000534","heartbeat_front_addr":"172.21.15.204:6805/1085000534","state":["exists","up"]},{"osd":7,"uuid":"a2e5a15c-8312-405a-ac02-f7e1fc9df2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6825","nonce":1337220609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6827","nonce":1337220609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6831","nonce":1337220609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6829","nonce":1337220609}]},"public_addr":"172.21.15.204:6825/1337220609","cluster_addr":"172.21.15.204:6827/1337220609","heartbeat_back_addr":"172.21.15.204:6831/1337220609","heartbeat_front_addr":"172.21.15.204:6829/1337220609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:49.984507+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.395306+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.632998+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.373627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.342920+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.438366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.331926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.406657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T15:00:45.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.915+0000 7f5987e00640 1 -- 172.21.15.143:0/532617193 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f596006b2c0 msgr2=0x7f596006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:45.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.915+0000 7f5987e00640 1 --2- 172.21.15.143:0/532617193 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f596006b2c0 0x7f596006d780 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5978009fd0 tx=0x7f597800d040 comp rx=0 tx=0).stop 2024-01-16T15:00:45.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.915+0000 7f5987e00640 1 -- 172.21.15.143:0/532617193 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59880b2c30 msgr2=0x7f5988145660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:45.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.915+0000 7f5987e00640 1 --2- 172.21.15.143:0/532617193 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59880b2c30 0x7f5988145660 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f597002c4e0 tx=0x7f5970001a00 comp rx=0 tx=0).stop 2024-01-16T15:00:45.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.915+0000 7f5987e00640 1 -- 172.21.15.143:0/532617193 shutdown_connections 2024-01-16T15:00:45.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.915+0000 7f5987e00640 1 --2- 172.21.15.143:0/532617193 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f596006b2c0 0x7f596006d780 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:45.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.915+0000 7f5987e00640 1 --2- 172.21.15.143:0/532617193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5988145ba0 0x7f59880711d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:45.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.915+0000 7f5987e00640 1 --2- 172.21.15.143:0/532617193 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59880b2c30 0x7f5988145660 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:45.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.915+0000 7f5987e00640 1 --2- 172.21.15.143:0/532617193 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5968003ef0 0x7f598813e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:45.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.915+0000 7f5987e00640 1 -- 172.21.15.143:0/532617193 >> 172.21.15.143:0/532617193 conn(0x7f59880cb830 msgr2=0x7f5988071e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:45.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.915+0000 7f5987e00640 1 -- 172.21.15.143:0/532617193 shutdown_connections 2024-01-16T15:00:45.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:45.915+0000 7f5987e00640 1 -- 172.21.15.143:0/532617193 wait complete. 2024-01-16T15:00:45.915 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-16T14:59:51.727720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-16T15:00:01.010472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-16T15:00:02.041228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-16T15:00:45.916 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-16T15:00:46.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.148+0000 7f5dbda00640 1 Processor -- start 2024-01-16T15:00:46.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.148+0000 7f5dbda00640 1 -- start start 2024-01-16T15:00:46.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.150+0000 7f5dbda00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5db8072160 0x7f5db815e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:46.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.150+0000 7f5dbda00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5db815ec90 con 0x7f5db80c49f0 2024-01-16T15:00:46.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.150+0000 7f5dbda00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5db80cd530 con 0x7f5db806de10 2024-01-16T15:00:46.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.150+0000 7f5dbda00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db80c4ed0 con 0x7f5db8072160 2024-01-16T15:00:46.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.150+0000 7f5dade00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f5db80c49f0 0x7f5db806d700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50534/0 (socket says 172.21.15.143:50534) 2024-01-16T15:00:46.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.150+0000 7f5dade00640 1 -- 172.21.15.143:0/1871071356 learned_addr learned my addr 172.21.15.143:0/1871071356 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:46.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.151+0000 7f5db6a00640 1 --2- 172.21.15.143:0/1871071356 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5db8072160 0x7f5db815e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:46.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.151+0000 7f5db6000640 1 -- 172.21.15.143:0/1871071356 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 512067173 0 0) 0x7f5da0001110 con 0x7f5db80c49f0 2024-01-16T15:00:46.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.151+0000 7f5db6000640 1 -- 172.21.15.143:0/1871071356 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d98003520 con 0x7f5db80c49f0 2024-01-16T15:00:46.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.151+0000 7f5db6000640 1 -- 172.21.15.143:0/1871071356 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1473402964 0 0) 0x7f5d98003520 con 0x7f5db80c49f0 2024-01-16T15:00:46.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.151+0000 7f5db6000640 1 -- 172.21.15.143:0/1871071356 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5db8072160 msgr2=0x7f5db815e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:46.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.151+0000 7f5db6000640 1 --2- 172.21.15.143:0/1871071356 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5db8072160 0x7f5db815e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:46.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.151+0000 7f5db6000640 1 -- 172.21.15.143:0/1871071356 >> v1:172.21.15.204:6789/0 conn(0x7f5db806de10 legacy=0x7f5db80705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:46.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.151+0000 7f5db6000640 1 -- 172.21.15.143:0/1871071356 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db80c51b0 con 0x7f5db80c49f0 2024-01-16T15:00:46.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.151+0000 7f5db6000640 1 -- 172.21.15.143:0/1871071356 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7f5da0001800 con 0x7f5db80c49f0 2024-01-16T15:00:46.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.151+0000 7f5db6000640 1 -- 172.21.15.143:0/1871071356 >> v1:172.21.15.143:6789/0 conn(0x7f5db80c49f0 legacy=0x7f5db806d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:46.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.151+0000 7f5db6000640 1 --2- 172.21.15.143:0/1871071356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d980039a0 0x7f5d98003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:46.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.151+0000 7f5db6000640 1 --2- 172.21.15.143:0/1871071356 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5db8072160 0x7f5d980047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:46.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.152+0000 7f5db6000640 1 --2- 172.21.15.143:0/1871071356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d98004d00 0x7f5d98005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:46.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.152+0000 7f5db6000640 1 -- 172.21.15.143:0/1871071356 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db80c4ed0 con 0x7f5d98004d00 2024-01-16T15:00:46.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.152+0000 7f5db6000640 1 -- 172.21.15.143:0/1871071356 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9800dc70 con 0x7f5d980039a0 2024-01-16T15:00:46.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.152+0000 7f5db6000640 1 -- 172.21.15.143:0/1871071356 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9800de30 con 0x7f5db8072160 2024-01-16T15:00:46.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.152+0000 7f5db7400640 1 --2- 172.21.15.143:0/1871071356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d98004d00 0x7f5d98005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:46.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.152+0000 7f5db6a00640 1 --2- 172.21.15.143:0/1871071356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d980039a0 0x7f5d98003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:46.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.152+0000 7f5dade00640 1 --2- 172.21.15.143:0/1871071356 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5db8072160 0x7f5d980047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:46.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.152+0000 7f5db7400640 1 -- 172.21.15.143:0/1871071356 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5db8072160 msgr2=0x7f5d980047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:46.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.152+0000 7f5db7400640 1 --2- 172.21.15.143:0/1871071356 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5db8072160 0x7f5d980047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:46.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.152+0000 7f5db7400640 1 -- 172.21.15.143:0/1871071356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d980039a0 msgr2=0x7f5d98003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:46.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.152+0000 7f5db7400640 1 --2- 172.21.15.143:0/1871071356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d980039a0 0x7f5d98003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:46.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.152+0000 7f5db7400640 1 -- 172.21.15.143:0/1871071356 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d98003520 con 0x7f5d98004d00 2024-01-16T15:00:46.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.153+0000 7f5db7400640 1 --2- 172.21.15.143:0/1871071356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d98004d00 0x7f5d98005180 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5db80b51a0 tx=0x7f5d980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f51c628b60a5ac62 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:46.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.153+0000 7f5db6000640 1 -- 172.21.15.143:0/1871071356 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db0003b70 con 0x7f5d98004d00 2024-01-16T15:00:46.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.154+0000 7f5db6000640 1 -- 172.21.15.143:0/1871071356 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db0003cd0 con 0x7f5d98004d00 2024-01-16T15:00:46.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.154+0000 7f5db6000640 1 -- 172.21.15.143:0/1871071356 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db0005310 con 0x7f5d98004d00 2024-01-16T15:00:46.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.154+0000 7f5dbda00640 1 -- 172.21.15.143:0/1871071356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d98004d00 msgr2=0x7f5d98005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:46.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.154+0000 7f5dbda00640 1 --2- 172.21.15.143:0/1871071356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d98004d00 0x7f5d98005180 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5db80b51a0 tx=0x7f5d980057f0 comp rx=0 tx=0).stop 2024-01-16T15:00:46.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.154+0000 7f5db7400640 1 -- 172.21.15.143:0/1871071356 reap_dead start 2024-01-16T15:00:46.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.154+0000 7f5dbda00640 1 -- 172.21.15.143:0/1871071356 shutdown_connections 2024-01-16T15:00:46.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.154+0000 7f5dbda00640 1 -- 172.21.15.143:0/1871071356 >> 172.21.15.143:0/1871071356 conn(0x7f5db80b0c50 msgr2=0x7f5db80b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:46.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.154+0000 7f5dbda00640 1 -- 172.21.15.143:0/1871071356 shutdown_connections 2024-01-16T15:00:46.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.154+0000 7f5dbda00640 1 -- 172.21.15.143:0/1871071356 wait complete. 2024-01-16T15:00:46.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.156+0000 7f5dbda00640 1 Processor -- start 2024-01-16T15:00:46.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.156+0000 7f5dbda00640 1 -- start start 2024-01-16T15:00:46.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.157+0000 7f5dbda00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d98004d00 0x7f5db815a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:46.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.157+0000 7f5dbda00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5db814caf0 0x7f5db809bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:46.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.157+0000 7f5dbda00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5db809c480 0x7f5db80cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:46.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.157+0000 7f5dbda00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db80c7210 con 0x7f5db809c480 2024-01-16T15:00:46.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.157+0000 7f5dbda00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db80c74f0 con 0x7f5db814caf0 2024-01-16T15:00:46.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.157+0000 7f5dbda00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db80c77d0 con 0x7f5d98004d00 2024-01-16T15:00:46.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.157+0000 7f5db7400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5db809c480 0x7f5db80cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:46.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.157+0000 7f5db7400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5db809c480 0x7f5db80cb2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59752/0 (socket says 172.21.15.143:59752) 2024-01-16T15:00:46.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.157+0000 7f5db7400640 1 -- 172.21.15.143:0/1097451044 learned_addr learned my addr 172.21.15.143:0/1097451044 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:46.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.158+0000 7f5dade00640 1 --2- 172.21.15.143:0/1097451044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5db814caf0 0x7f5db809bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:46.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.158+0000 7f5db6a00640 1 --2- 172.21.15.143:0/1097451044 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d98004d00 0x7f5db815a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:46.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.158+0000 7f5db7400640 1 -- 172.21.15.143:0/1097451044 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d98004d00 msgr2=0x7f5db815a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:46.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.158+0000 7f5db7400640 1 --2- 172.21.15.143:0/1097451044 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d98004d00 0x7f5db815a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:46.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.158+0000 7f5db7400640 1 -- 172.21.15.143:0/1097451044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5db814caf0 msgr2=0x7f5db809bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:46.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.158+0000 7f5db7400640 1 --2- 172.21.15.143:0/1097451044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5db814caf0 0x7f5db809bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:46.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.158+0000 7f5db7400640 1 -- 172.21.15.143:0/1097451044 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db80c7ab0 con 0x7f5db809c480 2024-01-16T15:00:46.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.158+0000 7f5db6a00640 1 --2- 172.21.15.143:0/1097451044 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d98004d00 0x7f5db815a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:46.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.158+0000 7f5db7400640 1 --2- 172.21.15.143:0/1097451044 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5db809c480 0x7f5db80cb2d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5db806ead0 tx=0x7f5db00014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:46.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.158+0000 7f5daf400640 1 -- 172.21.15.143:0/1097451044 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db0006d50 con 0x7f5db809c480 2024-01-16T15:00:46.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.158+0000 7f5daf400640 1 -- 172.21.15.143:0/1097451044 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db0003e40 con 0x7f5db809c480 2024-01-16T15:00:46.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.158+0000 7f5daf400640 1 -- 172.21.15.143:0/1097451044 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db0007230 con 0x7f5db809c480 2024-01-16T15:00:46.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.158+0000 7f5dbda00640 1 -- 172.21.15.143:0/1097451044 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5db80c7d90 con 0x7f5db809c480 2024-01-16T15:00:46.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.159+0000 7f5dbda00640 1 -- 172.21.15.143:0/1097451044 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5db8159e60 con 0x7f5db809c480 2024-01-16T15:00:46.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.160+0000 7f5dbda00640 1 -- 172.21.15.143:0/1097451044 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5db804e8b0 con 0x7f5db809c480 2024-01-16T15:00:46.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.161+0000 7f5daf400640 1 -- 172.21.15.143:0/1097451044 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f5db0041020 con 0x7f5db809c480 2024-01-16T15:00:46.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.161+0000 7f5daf400640 1 --2- 172.21.15.143:0/1097451044 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f5d9006b2c0 0x7f5d9006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:46.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.162+0000 7f5daf400640 1 -- 172.21.15.143:0/1097451044 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f5db00abd20 con 0x7f5db809c480 2024-01-16T15:00:46.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.162+0000 7f5db6a00640 1 --2- 172.21.15.143:0/1097451044 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f5d9006b2c0 0x7f5d9006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:46.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.162+0000 7f5db6a00640 1 --2- 172.21.15.143:0/1097451044 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f5d9006b2c0 0x7f5d9006d780 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5da8009a00 tx=0x7f5da8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:46.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.166+0000 7f5daf400640 1 -- 172.21.15.143:0/1097451044 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5db00769f0 con 0x7f5db809c480 2024-01-16T15:00:46.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.355+0000 7f5dbda00640 1 -- 172.21.15.143:0/1097451044 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5db809cec0 con 0x7f5db809c480 2024-01-16T15:00:46.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.355+0000 7f5daf400640 1 -- 172.21.15.143:0/1097451044 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5db00490a0 con 0x7f5db809c480 2024-01-16T15:00:46.335 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-16T15:00:46.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.358+0000 7f5dbda00640 1 -- 172.21.15.143:0/1097451044 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f5d9006b2c0 msgr2=0x7f5d9006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:46.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.358+0000 7f5dbda00640 1 --2- 172.21.15.143:0/1097451044 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f5d9006b2c0 0x7f5d9006d780 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5da8009a00 tx=0x7f5da8008040 comp rx=0 tx=0).stop 2024-01-16T15:00:46.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.358+0000 7f5dbda00640 1 -- 172.21.15.143:0/1097451044 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5db809c480 msgr2=0x7f5db80cb2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:46.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.358+0000 7f5dbda00640 1 --2- 172.21.15.143:0/1097451044 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5db809c480 0x7f5db80cb2d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5db806ead0 tx=0x7f5db00014a0 comp rx=0 tx=0).stop 2024-01-16T15:00:46.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.358+0000 7f5dbda00640 1 -- 172.21.15.143:0/1097451044 shutdown_connections 2024-01-16T15:00:46.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.358+0000 7f5dbda00640 1 --2- 172.21.15.143:0/1097451044 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f5d9006b2c0 0x7f5d9006d780 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:46.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.358+0000 7f5dbda00640 1 --2- 172.21.15.143:0/1097451044 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5db809c480 0x7f5db80cb2d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:46.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.358+0000 7f5dbda00640 1 --2- 172.21.15.143:0/1097451044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5db814caf0 0x7f5db809bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:46.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.358+0000 7f5dbda00640 1 --2- 172.21.15.143:0/1097451044 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d98004d00 0x7f5db815a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:46.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.358+0000 7f5dbda00640 1 -- 172.21.15.143:0/1097451044 >> 172.21.15.143:0/1097451044 conn(0x7f5db80b0c50 msgr2=0x7f5db80704c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:46.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.358+0000 7f5dbda00640 1 -- 172.21.15.143:0/1097451044 shutdown_connections 2024-01-16T15:00:46.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.359+0000 7f5dbda00640 1 -- 172.21.15.143:0/1097451044 wait complete. 2024-01-16T15:00:46.358 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-16T15:00:46.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.592+0000 7f6a80200640 1 Processor -- start 2024-01-16T15:00:46.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.592+0000 7f6a80200640 1 -- start start 2024-01-16T15:00:46.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.594+0000 7f6a80200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6a7815b440 0x7f6a780cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:46.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.594+0000 7f6a80200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a780cbf20 0x7f6a780c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:46.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.594+0000 7f6a80200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a7815eea0 con 0x7f6a780c57e0 2024-01-16T15:00:46.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.594+0000 7f6a80200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a780cc3d0 con 0x7f6a7815b440 2024-01-16T15:00:46.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.594+0000 7f6a80200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7815b820 con 0x7f6a780cbf20 2024-01-16T15:00:46.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.594+0000 7f6a7d800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6a7815b440 0x7f6a780cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:46.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.594+0000 7f6a7d800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6a7815b440 0x7f6a780cb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59760/0 (socket says 172.21.15.143:59760) 2024-01-16T15:00:46.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.594+0000 7f6a7d800640 1 -- 172.21.15.143:0/1760265943 learned_addr learned my addr 172.21.15.143:0/1760265943 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:46.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.594+0000 7f6a7ce00640 1 --2- 172.21.15.143:0/1760265943 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a780cbf20 0x7f6a780c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:46.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.595+0000 7f6a7d800640 1 -- 172.21.15.143:0/1760265943 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a780cbf20 msgr2=0x7f6a780c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:46.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.595+0000 7f6a7d800640 1 --2- 172.21.15.143:0/1760265943 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a780cbf20 0x7f6a780c52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:46.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.595+0000 7f6a7d800640 1 -- 172.21.15.143:0/1760265943 >> v1:172.21.15.204:6789/0 conn(0x7f6a780c57e0 legacy=0x7f6a780c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:46.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.595+0000 7f6a7d800640 1 -- 172.21.15.143:0/1760265943 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7806d600 con 0x7f6a7815b440 2024-01-16T15:00:46.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.596+0000 7f6a7d800640 1 --2- 172.21.15.143:0/1760265943 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a7815b440 0x7f6a780cb9e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6a68009870 tx=0x7f6a6802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2769e9ba97725e1 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:46.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.596+0000 7f6a77e00640 1 -- 172.21.15.143:0/1760265943 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a68031640 con 0x7f6a7815b440 2024-01-16T15:00:46.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.596+0000 7f6a77e00640 1 -- 172.21.15.143:0/1760265943 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a680317a0 con 0x7f6a7815b440 2024-01-16T15:00:46.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.596+0000 7f6a77e00640 1 -- 172.21.15.143:0/1760265943 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a68031b30 con 0x7f6a7815b440 2024-01-16T15:00:46.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.597+0000 7f6a80200640 1 -- 172.21.15.143:0/1760265943 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a7815b440 msgr2=0x7f6a780cb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:46.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.597+0000 7f6a80200640 1 --2- 172.21.15.143:0/1760265943 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a7815b440 0x7f6a780cb9e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6a68009870 tx=0x7f6a6802b8f0 comp rx=0 tx=0).stop 2024-01-16T15:00:46.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.597+0000 7f6a80200640 1 -- 172.21.15.143:0/1760265943 shutdown_connections 2024-01-16T15:00:46.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.597+0000 7f6a80200640 1 --2- 172.21.15.143:0/1760265943 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a780cbf20 0x7f6a780c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:46.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.597+0000 7f6a80200640 1 --2- 172.21.15.143:0/1760265943 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a7815b440 0x7f6a780cb9e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:46.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.597+0000 7f6a80200640 1 -- 172.21.15.143:0/1760265943 >> 172.21.15.143:0/1760265943 conn(0x7f6a780b0c50 msgr2=0x7f6a780b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:46.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.597+0000 7f6a80200640 1 -- 172.21.15.143:0/1760265943 shutdown_connections 2024-01-16T15:00:46.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.597+0000 7f6a80200640 1 -- 172.21.15.143:0/1760265943 wait complete. 2024-01-16T15:00:46.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.599+0000 7f6a80200640 1 Processor -- start 2024-01-16T15:00:46.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.599+0000 7f6a80200640 1 -- start start 2024-01-16T15:00:46.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.600+0000 7f6a80200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a780c57e0 0x7f6a780c8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:46.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.600+0000 7f6a80200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a780cbf20 0x7f6a780c60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:46.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.600+0000 7f6a80200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a7815b440 0x7f6a780c6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:46.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.600+0000 7f6a80200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a780c8ef0 con 0x7f6a780c57e0 2024-01-16T15:00:46.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.600+0000 7f6a80200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a78157690 con 0x7f6a7815b440 2024-01-16T15:00:46.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.600+0000 7f6a80200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a78157970 con 0x7f6a780cbf20 2024-01-16T15:00:46.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.600+0000 7f6a7d800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a780c57e0 0x7f6a780c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:46.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.600+0000 7f6a7d800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a780c57e0 0x7f6a780c8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59772/0 (socket says 172.21.15.143:59772) 2024-01-16T15:00:46.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.600+0000 7f6a7d800640 1 -- 172.21.15.143:0/3073015778 learned_addr learned my addr 172.21.15.143:0/3073015778 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:46.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.600+0000 7f6a7e200640 1 --2- 172.21.15.143:0/3073015778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a7815b440 0x7f6a780c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:46.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.600+0000 7f6a7d800640 1 -- 172.21.15.143:0/3073015778 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a780cbf20 msgr2=0x7f6a780c60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:46.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.600+0000 7f6a7ce00640 1 --2- 172.21.15.143:0/3073015778 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a780cbf20 0x7f6a780c60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:46.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.600+0000 7f6a7d800640 1 --2- 172.21.15.143:0/3073015778 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a780cbf20 0x7f6a780c60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:46.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.600+0000 7f6a7d800640 1 -- 172.21.15.143:0/3073015778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a7815b440 msgr2=0x7f6a780c6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:46.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.600+0000 7f6a7d800640 1 --2- 172.21.15.143:0/3073015778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a7815b440 0x7f6a780c6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:46.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.600+0000 7f6a7d800640 1 -- 172.21.15.143:0/3073015778 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a680094d0 con 0x7f6a780c57e0 2024-01-16T15:00:46.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.600+0000 7f6a7e200640 1 --2- 172.21.15.143:0/3073015778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a7815b440 0x7f6a780c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:46.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.601+0000 7f6a7d800640 1 --2- 172.21.15.143:0/3073015778 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a780c57e0 0x7f6a780c8910 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6a68009870 tx=0x7f6a680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:46.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.601+0000 7f6a75600640 1 -- 172.21.15.143:0/3073015778 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6803ac60 con 0x7f6a780c57e0 2024-01-16T15:00:46.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.601+0000 7f6a75600640 1 -- 172.21.15.143:0/3073015778 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a6803adc0 con 0x7f6a780c57e0 2024-01-16T15:00:46.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.601+0000 7f6a80200640 1 -- 172.21.15.143:0/3073015778 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a78157c50 con 0x7f6a780c57e0 2024-01-16T15:00:46.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.601+0000 7f6a75600640 1 -- 172.21.15.143:0/3073015778 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a68044630 con 0x7f6a780c57e0 2024-01-16T15:00:46.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.601+0000 7f6a80200640 1 -- 172.21.15.143:0/3073015778 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a7815dc20 con 0x7f6a780c57e0 2024-01-16T15:00:46.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.602+0000 7f6a80200640 1 -- 172.21.15.143:0/3073015778 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a7804e8b0 con 0x7f6a780c57e0 2024-01-16T15:00:46.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.603+0000 7f6a75600640 1 -- 172.21.15.143:0/3073015778 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f6a6802f050 con 0x7f6a780c57e0 2024-01-16T15:00:46.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.604+0000 7f6a75600640 1 --2- 172.21.15.143:0/3073015778 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6a5006b2c0 0x7f6a5006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:46.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.604+0000 7f6a75600640 1 -- 172.21.15.143:0/3073015778 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f6a680aba30 con 0x7f6a780c57e0 2024-01-16T15:00:46.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.605+0000 7f6a7ce00640 1 --2- 172.21.15.143:0/3073015778 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6a5006b2c0 0x7f6a5006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:46.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.605+0000 7f6a7ce00640 1 --2- 172.21.15.143:0/3073015778 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6a5006b2c0 0x7f6a5006d780 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6a781f6bc0 tx=0x7f6a60008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:46.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.609+0000 7f6a75600640 1 -- 172.21.15.143:0/3073015778 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6a680768b0 con 0x7f6a780c57e0 2024-01-16T15:00:46.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.805+0000 7f6a80200640 1 -- 172.21.15.143:0/3073015778 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6a7806dc80 con 0x7f6a780c57e0 2024-01-16T15:00:46.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.806+0000 7f6a75600640 1 -- 172.21.15.143:0/3073015778 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6a68031e60 con 0x7f6a780c57e0 2024-01-16T15:00:46.786 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2024-01-16T15:00:46.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.808+0000 7f6a80200640 1 -- 172.21.15.143:0/3073015778 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6a5006b2c0 msgr2=0x7f6a5006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:46.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.808+0000 7f6a80200640 1 --2- 172.21.15.143:0/3073015778 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6a5006b2c0 0x7f6a5006d780 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6a781f6bc0 tx=0x7f6a60008040 comp rx=0 tx=0).stop 2024-01-16T15:00:46.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.808+0000 7f6a80200640 1 -- 172.21.15.143:0/3073015778 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a780c57e0 msgr2=0x7f6a780c8910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:46.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.808+0000 7f6a80200640 1 --2- 172.21.15.143:0/3073015778 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a780c57e0 0x7f6a780c8910 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6a68009870 tx=0x7f6a680046c0 comp rx=0 tx=0).stop 2024-01-16T15:00:46.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.808+0000 7f6a80200640 1 -- 172.21.15.143:0/3073015778 shutdown_connections 2024-01-16T15:00:46.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.808+0000 7f6a80200640 1 --2- 172.21.15.143:0/3073015778 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6a5006b2c0 0x7f6a5006d780 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:46.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.808+0000 7f6a80200640 1 --2- 172.21.15.143:0/3073015778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a7815b440 0x7f6a780c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:46.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.808+0000 7f6a80200640 1 --2- 172.21.15.143:0/3073015778 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a780cbf20 0x7f6a780c60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:46.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.808+0000 7f6a80200640 1 --2- 172.21.15.143:0/3073015778 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a780c57e0 0x7f6a780c8910 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:46.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.808+0000 7f6a80200640 1 -- 172.21.15.143:0/3073015778 >> 172.21.15.143:0/3073015778 conn(0x7f6a780b0c50 msgr2=0x7f6a7806ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:46.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.808+0000 7f6a80200640 1 -- 172.21.15.143:0/3073015778 shutdown_connections 2024-01-16T15:00:46.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:46.808+0000 7f6a80200640 1 -- 172.21.15.143:0/3073015778 wait complete. 2024-01-16T15:00:46.807 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-16T15:00:47.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.038+0000 7fe129200640 1 Processor -- start 2024-01-16T15:00:47.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.038+0000 7fe129200640 1 -- start start 2024-01-16T15:00:47.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.040+0000 7fe129200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe12415d3f0 0x7fe1240d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:47.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.040+0000 7fe129200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe12415d7d0 0x7fe1240d1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:47.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.040+0000 7fe129200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe1240d0e40 0x7fe1240afea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:47.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.040+0000 7fe129200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1240d15e0 con 0x7fe12415d7d0 2024-01-16T15:00:47.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.040+0000 7fe129200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1240d0280 con 0x7fe1240d0e40 2024-01-16T15:00:47.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.040+0000 7fe129200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1240d1300 con 0x7fe12415d3f0 2024-01-16T15:00:47.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.040+0000 7fe11be00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe12415d7d0 0x7fe1240d1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:47.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.040+0000 7fe11be00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe12415d7d0 0x7fe1240d1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59778/0 (socket says 172.21.15.143:59778) 2024-01-16T15:00:47.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.040+0000 7fe11be00640 1 -- 172.21.15.143:0/284956826 learned_addr learned my addr 172.21.15.143:0/284956826 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:47.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.040+0000 7fe123400640 1 --2- 172.21.15.143:0/284956826 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe12415d3f0 0x7fe1240d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:47.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.040+0000 7fe123e00640 1 --2- 172.21.15.143:0/284956826 >> v2:172.21.15.204:3300/0 conn(0x7fe1240d0e40 0x7fe1240afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:47.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.041+0000 7fe11be00640 1 -- 172.21.15.143:0/284956826 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe12415d3f0 msgr2=0x7fe1240d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:47.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.041+0000 7fe11be00640 1 --2- 172.21.15.143:0/284956826 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe12415d3f0 0x7fe1240d1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:47.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.041+0000 7fe11be00640 1 -- 172.21.15.143:0/284956826 >> v2:172.21.15.204:3300/0 conn(0x7fe1240d0e40 msgr2=0x7fe1240afea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:47.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.041+0000 7fe11be00640 1 --2- 172.21.15.143:0/284956826 >> v2:172.21.15.204:3300/0 conn(0x7fe1240d0e40 0x7fe1240afea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:47.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.041+0000 7fe11be00640 1 -- 172.21.15.143:0/284956826 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1240cfc30 con 0x7fe12415d7d0 2024-01-16T15:00:47.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.042+0000 7fe11be00640 1 --2- 172.21.15.143:0/284956826 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe12415d7d0 0x7fe1240d1d70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe1240d0dd0 tx=0x7fe10c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6971baa22d628d3b server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:47.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.042+0000 7fe122a00640 1 -- 172.21.15.143:0/284956826 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c031870 con 0x7fe12415d7d0 2024-01-16T15:00:47.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.042+0000 7fe122a00640 1 -- 172.21.15.143:0/284956826 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe10c0319d0 con 0x7fe12415d7d0 2024-01-16T15:00:47.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.042+0000 7fe122a00640 1 -- 172.21.15.143:0/284956826 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c031d60 con 0x7fe12415d7d0 2024-01-16T15:00:47.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.043+0000 7fe129200640 1 -- 172.21.15.143:0/284956826 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe12415d7d0 msgr2=0x7fe1240d1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:47.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.043+0000 7fe129200640 1 --2- 172.21.15.143:0/284956826 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe12415d7d0 0x7fe1240d1d70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe1240d0dd0 tx=0x7fe10c02b910 comp rx=0 tx=0).stop 2024-01-16T15:00:47.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.043+0000 7fe129200640 1 -- 172.21.15.143:0/284956826 shutdown_connections 2024-01-16T15:00:47.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.043+0000 7fe129200640 1 --2- 172.21.15.143:0/284956826 >> v2:172.21.15.204:3300/0 conn(0x7fe1240d0e40 0x7fe1240afea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:47.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.043+0000 7fe129200640 1 --2- 172.21.15.143:0/284956826 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe12415d7d0 0x7fe1240d1d70 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:47.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.043+0000 7fe129200640 1 --2- 172.21.15.143:0/284956826 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe12415d3f0 0x7fe1240d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:47.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.043+0000 7fe129200640 1 -- 172.21.15.143:0/284956826 >> 172.21.15.143:0/284956826 conn(0x7fe12409c5b0 msgr2=0x7fe1240d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:47.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.043+0000 7fe129200640 1 -- 172.21.15.143:0/284956826 shutdown_connections 2024-01-16T15:00:47.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.043+0000 7fe129200640 1 -- 172.21.15.143:0/284956826 wait complete. 2024-01-16T15:00:47.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.045+0000 7fe129200640 1 Processor -- start 2024-01-16T15:00:47.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.045+0000 7fe129200640 1 -- start start 2024-01-16T15:00:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.046+0000 7fe129200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1240d0e40 0x7fe1240b2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.046+0000 7fe129200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe12415d3f0 0x7fe1240b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.046+0000 7fe129200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe12415d7d0 0x7fe124153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.046+0000 7fe129200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1240b4b30 con 0x7fe1240d0e40 2024-01-16T15:00:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.046+0000 7fe129200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe124153590 con 0x7fe12415d7d0 2024-01-16T15:00:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.046+0000 7fe129200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1240b2f50 con 0x7fe12415d3f0 2024-01-16T15:00:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.046+0000 7fe123400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1240d0e40 0x7fe1240b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.046+0000 7fe123400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1240d0e40 0x7fe1240b2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59780/0 (socket says 172.21.15.143:59780) 2024-01-16T15:00:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.046+0000 7fe123400640 1 -- 172.21.15.143:0/3218079063 learned_addr learned my addr 172.21.15.143:0/3218079063 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.046+0000 7fe11be00640 1 --2- 172.21.15.143:0/3218079063 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe12415d3f0 0x7fe1240b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.046+0000 7fe123e00640 1 --2- 172.21.15.143:0/3218079063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe12415d7d0 0x7fe124153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.046+0000 7fe123400640 1 -- 172.21.15.143:0/3218079063 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe12415d3f0 msgr2=0x7fe1240b45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.046+0000 7fe123400640 1 --2- 172.21.15.143:0/3218079063 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe12415d3f0 0x7fe1240b45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:47.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.046+0000 7fe123400640 1 -- 172.21.15.143:0/3218079063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe12415d7d0 msgr2=0x7fe124153050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:47.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.046+0000 7fe123400640 1 --2- 172.21.15.143:0/3218079063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe12415d7d0 0x7fe124153050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:47.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.046+0000 7fe123400640 1 -- 172.21.15.143:0/3218079063 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10c0094d0 con 0x7fe1240d0e40 2024-01-16T15:00:47.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.047+0000 7fe123400640 1 --2- 172.21.15.143:0/3218079063 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1240d0e40 0x7fe1240b2970 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe11400d800 tx=0x7fe11400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:47.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.047+0000 7fe11b400640 1 -- 172.21.15.143:0/3218079063 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11400b6c0 con 0x7fe1240d0e40 2024-01-16T15:00:47.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.047+0000 7fe11b400640 1 -- 172.21.15.143:0/3218079063 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe11400b820 con 0x7fe1240d0e40 2024-01-16T15:00:47.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.047+0000 7fe129200640 1 -- 172.21.15.143:0/3218079063 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1240b3230 con 0x7fe1240d0e40 2024-01-16T15:00:47.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.047+0000 7fe11b400640 1 -- 172.21.15.143:0/3218079063 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11400bbb0 con 0x7fe1240d0e40 2024-01-16T15:00:47.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.047+0000 7fe129200640 1 -- 172.21.15.143:0/3218079063 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1240b37f0 con 0x7fe1240d0e40 2024-01-16T15:00:47.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.048+0000 7fe129200640 1 -- 172.21.15.143:0/3218079063 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe12404e840 con 0x7fe1240d0e40 2024-01-16T15:00:47.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.049+0000 7fe11b400640 1 -- 172.21.15.143:0/3218079063 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fe11400bd10 con 0x7fe1240d0e40 2024-01-16T15:00:47.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.050+0000 7fe11b400640 1 --2- 172.21.15.143:0/3218079063 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe0fc06b2c0 0x7fe0fc06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:47.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.050+0000 7fe11b400640 1 -- 172.21.15.143:0/3218079063 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fe11408a160 con 0x7fe1240d0e40 2024-01-16T15:00:47.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.051+0000 7fe11be00640 1 --2- 172.21.15.143:0/3218079063 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe0fc06b2c0 0x7fe0fc06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:47.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.051+0000 7fe11be00640 1 --2- 172.21.15.143:0/3218079063 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe0fc06b2c0 0x7fe0fc06d780 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe10c036000 tx=0x7fe10c03c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:47.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.055+0000 7fe11b400640 1 -- 172.21.15.143:0/3218079063 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe114054f40 con 0x7fe1240d0e40 2024-01-16T15:00:47.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.244+0000 7fe129200640 1 -- 172.21.15.143:0/3218079063 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe1240b7360 con 0x7fe1240d0e40 2024-01-16T15:00:47.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.244+0000 7fe11b400640 1 -- 172.21.15.143:0/3218079063 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe114058560 con 0x7fe1240d0e40 2024-01-16T15:00:47.224 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2024-01-16T15:00:47.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.247+0000 7fe129200640 1 -- 172.21.15.143:0/3218079063 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe0fc06b2c0 msgr2=0x7fe0fc06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:47.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.247+0000 7fe129200640 1 --2- 172.21.15.143:0/3218079063 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe0fc06b2c0 0x7fe0fc06d780 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe10c036000 tx=0x7fe10c03c040 comp rx=0 tx=0).stop 2024-01-16T15:00:47.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.247+0000 7fe129200640 1 -- 172.21.15.143:0/3218079063 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1240d0e40 msgr2=0x7fe1240b2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:47.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.247+0000 7fe129200640 1 --2- 172.21.15.143:0/3218079063 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1240d0e40 0x7fe1240b2970 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe11400d800 tx=0x7fe11400dcc0 comp rx=0 tx=0).stop 2024-01-16T15:00:47.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.247+0000 7fe129200640 1 -- 172.21.15.143:0/3218079063 shutdown_connections 2024-01-16T15:00:47.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.247+0000 7fe129200640 1 --2- 172.21.15.143:0/3218079063 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe0fc06b2c0 0x7fe0fc06d780 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:47.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.247+0000 7fe129200640 1 --2- 172.21.15.143:0/3218079063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe12415d7d0 0x7fe124153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:47.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.247+0000 7fe129200640 1 --2- 172.21.15.143:0/3218079063 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe12415d3f0 0x7fe1240b45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:47.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.247+0000 7fe129200640 1 --2- 172.21.15.143:0/3218079063 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1240d0e40 0x7fe1240b2970 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:47.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.247+0000 7fe129200640 1 -- 172.21.15.143:0/3218079063 >> 172.21.15.143:0/3218079063 conn(0x7fe12409c5b0 msgr2=0x7fe1240b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:47.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.247+0000 7fe129200640 1 -- 172.21.15.143:0/3218079063 shutdown_connections 2024-01-16T15:00:47.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.247+0000 7fe129200640 1 -- 172.21.15.143:0/3218079063 wait complete. 2024-01-16T15:00:47.247 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-16T15:00:47.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.476+0000 7f08f9000640 1 Processor -- start 2024-01-16T15:00:47.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.477+0000 7f08f9000640 1 -- start start 2024-01-16T15:00:47.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.478+0000 7f08f9000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f08f415b440 0x7f08f40cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:47.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.478+0000 7f08f9000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08f40c59b0 0x7f08f406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:47.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.478+0000 7f08f9000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08f415eea0 con 0x7f08f40cbf20 2024-01-16T15:00:47.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.478+0000 7f08f9000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08f40c5e00 con 0x7f08f415b440 2024-01-16T15:00:47.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.478+0000 7f08f9000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f40cc3d0 con 0x7f08f40c59b0 2024-01-16T15:00:47.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.479+0000 7f08f2800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08f40c59b0 0x7f08f406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:47.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.479+0000 7f08f2800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08f40c59b0 0x7f08f406c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:39498/0 (socket says 172.21.15.143:39498) 2024-01-16T15:00:47.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.479+0000 7f08f2800640 1 -- 172.21.15.143:0/1999035308 learned_addr learned my addr 172.21.15.143:0/1999035308 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:47.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.479+0000 7f08f1e00640 1 --2- 172.21.15.143:0/1999035308 >> v2:172.21.15.204:3300/0 conn(0x7f08f415b440 0x7f08f40cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:47.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.480+0000 7f08f2800640 1 -- 172.21.15.143:0/1999035308 >> v2:172.21.15.204:3300/0 conn(0x7f08f415b440 msgr2=0x7f08f40cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:47.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.480+0000 7f08f2800640 1 --2- 172.21.15.143:0/1999035308 >> v2:172.21.15.204:3300/0 conn(0x7f08f415b440 0x7f08f40cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:47.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.480+0000 7f08f2800640 1 -- 172.21.15.143:0/1999035308 >> v1:172.21.15.204:6789/0 conn(0x7f08f40cbf20 legacy=0x7f08f40c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:47.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.480+0000 7f08f2800640 1 -- 172.21.15.143:0/1999035308 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f40abf30 con 0x7f08f40c59b0 2024-01-16T15:00:47.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.480+0000 7f08f0a00640 1 -- 172.21.15.143:0/1999035308 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 225027200 0 0) 0x7f08f415eea0 con 0x7f08f40cbf20 2024-01-16T15:00:47.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.481+0000 7f08f2800640 1 --2- 172.21.15.143:0/1999035308 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08f40c59b0 0x7f08f406c5f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f08ec009870 tx=0x7f08ec02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ed66f2284dd2588 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:47.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.482+0000 7f08f0a00640 1 -- 172.21.15.143:0/1999035308 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec031640 con 0x7f08f40c59b0 2024-01-16T15:00:47.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.482+0000 7f08f0a00640 1 -- 172.21.15.143:0/1999035308 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ec0317a0 con 0x7f08f40c59b0 2024-01-16T15:00:47.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.482+0000 7f08f0a00640 1 -- 172.21.15.143:0/1999035308 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec031b30 con 0x7f08f40c59b0 2024-01-16T15:00:47.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.482+0000 7f08f9000640 1 -- 172.21.15.143:0/1999035308 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08f40c59b0 msgr2=0x7f08f406c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:47.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.482+0000 7f08f9000640 1 --2- 172.21.15.143:0/1999035308 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08f40c59b0 0x7f08f406c5f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f08ec009870 tx=0x7f08ec02b8f0 comp rx=0 tx=0).stop 2024-01-16T15:00:47.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.482+0000 7f08f9000640 1 -- 172.21.15.143:0/1999035308 shutdown_connections 2024-01-16T15:00:47.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.482+0000 7f08f9000640 1 --2- 172.21.15.143:0/1999035308 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08f40c59b0 0x7f08f406c5f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:47.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.482+0000 7f08f9000640 1 --2- 172.21.15.143:0/1999035308 >> v2:172.21.15.204:3300/0 conn(0x7f08f415b440 0x7f08f40cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:47.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.482+0000 7f08f9000640 1 -- 172.21.15.143:0/1999035308 >> 172.21.15.143:0/1999035308 conn(0x7f08f40b0c50 msgr2=0x7f08f40b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:47.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.482+0000 7f08f9000640 1 -- 172.21.15.143:0/1999035308 shutdown_connections 2024-01-16T15:00:47.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.483+0000 7f08f9000640 1 -- 172.21.15.143:0/1999035308 wait complete. 2024-01-16T15:00:47.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.484+0000 7f08f9000640 1 Processor -- start 2024-01-16T15:00:47.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.484+0000 7f08f9000640 1 -- start start 2024-01-16T15:00:47.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.485+0000 7f08f9000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08f40c59b0 0x7f08f40aff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:47.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.485+0000 7f08f9000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08f40cbf20 0x7f08f4153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:47.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.485+0000 7f08f9000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08f415b440 0x7f08f4159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:47.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.485+0000 7f08f9000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f40b0b10 con 0x7f08f40cbf20 2024-01-16T15:00:47.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.485+0000 7f08f9000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f40af1d0 con 0x7f08f40c59b0 2024-01-16T15:00:47.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.485+0000 7f08f9000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f40b0830 con 0x7f08f415b440 2024-01-16T15:00:47.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.486+0000 7f08f1400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08f40cbf20 0x7f08f4153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:47.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.486+0000 7f08f1400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08f40cbf20 0x7f08f4153e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59796/0 (socket says 172.21.15.143:59796) 2024-01-16T15:00:47.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.486+0000 7f08f1400640 1 -- 172.21.15.143:0/3011014683 learned_addr learned my addr 172.21.15.143:0/3011014683 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:47.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.486+0000 7f08f2800640 1 --2- 172.21.15.143:0/3011014683 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08f415b440 0x7f08f4159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:47.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.486+0000 7f08f1e00640 1 --2- 172.21.15.143:0/3011014683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08f40c59b0 0x7f08f40aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:47.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.486+0000 7f08f1400640 1 -- 172.21.15.143:0/3011014683 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08f415b440 msgr2=0x7f08f4159470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:47.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.486+0000 7f08f1400640 1 --2- 172.21.15.143:0/3011014683 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08f415b440 0x7f08f4159470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:47.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.486+0000 7f08f1400640 1 -- 172.21.15.143:0/3011014683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08f40c59b0 msgr2=0x7f08f40aff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:47.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.486+0000 7f08f1400640 1 --2- 172.21.15.143:0/3011014683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08f40c59b0 0x7f08f40aff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:47.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.486+0000 7f08f1400640 1 -- 172.21.15.143:0/3011014683 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ec0094d0 con 0x7f08f40cbf20 2024-01-16T15:00:47.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.487+0000 7f08f1400640 1 --2- 172.21.15.143:0/3011014683 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08f40cbf20 0x7f08f4153e00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00c4b0 tx=0x7f08dc00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:47.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.487+0000 7f08ea000640 1 -- 172.21.15.143:0/3011014683 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc005480 con 0x7f08f40cbf20 2024-01-16T15:00:47.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.487+0000 7f08ea000640 1 -- 172.21.15.143:0/3011014683 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08dc0035f0 con 0x7f08f40cbf20 2024-01-16T15:00:47.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.487+0000 7f08f9000640 1 -- 172.21.15.143:0/3011014683 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08f40aeb10 con 0x7f08f40cbf20 2024-01-16T15:00:47.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.487+0000 7f08ea000640 1 -- 172.21.15.143:0/3011014683 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc0039e0 con 0x7f08f40cbf20 2024-01-16T15:00:47.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.487+0000 7f08f9000640 1 -- 172.21.15.143:0/3011014683 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08f40b0440 con 0x7f08f40cbf20 2024-01-16T15:00:47.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.488+0000 7f08f9000640 1 -- 172.21.15.143:0/3011014683 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08f404e8b0 con 0x7f08f40cbf20 2024-01-16T15:00:47.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.489+0000 7f08ea000640 1 -- 172.21.15.143:0/3011014683 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f08dc003b40 con 0x7f08f40cbf20 2024-01-16T15:00:47.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.490+0000 7f08ea000640 1 --2- 172.21.15.143:0/3011014683 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f08cc06b2c0 0x7f08cc06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:47.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.490+0000 7f08ea000640 1 -- 172.21.15.143:0/3011014683 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f08dc0891b0 con 0x7f08f40cbf20 2024-01-16T15:00:47.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.490+0000 7f08f1e00640 1 --2- 172.21.15.143:0/3011014683 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f08cc06b2c0 0x7f08cc06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:47.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.491+0000 7f08f1e00640 1 --2- 172.21.15.143:0/3011014683 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f08cc06b2c0 0x7f08cc06d780 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f08f4154d30 tx=0x7f08e400b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:47.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.494+0000 7f08ea000640 1 -- 172.21.15.143:0/3011014683 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f08dc054030 con 0x7f08f40cbf20 2024-01-16T15:00:47.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.698+0000 7f08f9000640 1 -- 172.21.15.143:0/3011014683 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f08f406dc80 con 0x7f08f40cbf20 2024-01-16T15:00:47.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.698+0000 7f08ea000640 1 -- 172.21.15.143:0/3011014683 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f08dc057650 con 0x7f08f40cbf20 2024-01-16T15:00:47.678 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2024-01-16T15:00:47.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.701+0000 7f08f9000640 1 -- 172.21.15.143:0/3011014683 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f08cc06b2c0 msgr2=0x7f08cc06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:47.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.701+0000 7f08f9000640 1 --2- 172.21.15.143:0/3011014683 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f08cc06b2c0 0x7f08cc06d780 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f08f4154d30 tx=0x7f08e400b040 comp rx=0 tx=0).stop 2024-01-16T15:00:47.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.701+0000 7f08f9000640 1 -- 172.21.15.143:0/3011014683 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08f40cbf20 msgr2=0x7f08f4153e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:47.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.701+0000 7f08f9000640 1 --2- 172.21.15.143:0/3011014683 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08f40cbf20 0x7f08f4153e00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00c4b0 tx=0x7f08dc00c970 comp rx=0 tx=0).stop 2024-01-16T15:00:47.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.701+0000 7f08f9000640 1 -- 172.21.15.143:0/3011014683 shutdown_connections 2024-01-16T15:00:47.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.701+0000 7f08f9000640 1 --2- 172.21.15.143:0/3011014683 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f08cc06b2c0 0x7f08cc06d780 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:47.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.701+0000 7f08f9000640 1 --2- 172.21.15.143:0/3011014683 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08f415b440 0x7f08f4159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:47.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.701+0000 7f08f9000640 1 --2- 172.21.15.143:0/3011014683 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08f40cbf20 0x7f08f4153e00 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:47.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.701+0000 7f08f9000640 1 --2- 172.21.15.143:0/3011014683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08f40c59b0 0x7f08f40aff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:47.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.701+0000 7f08f9000640 1 -- 172.21.15.143:0/3011014683 >> 172.21.15.143:0/3011014683 conn(0x7f08f40b0c50 msgr2=0x7f08f41581a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:47.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.701+0000 7f08f9000640 1 -- 172.21.15.143:0/3011014683 shutdown_connections 2024-01-16T15:00:47.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:47.701+0000 7f08f9000640 1 -- 172.21.15.143:0/3011014683 wait complete. 2024-01-16T15:00:47.700 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-16T15:00:47.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:47.700 DEBUG:teuthology.orchestra.run.smithi143:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-01-16T15:00:47.799 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2024-01-16T15:00:47.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:47.800 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-16T15:00:47.854 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-16T15:00:47.854 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T15:00:47.911 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-16T15:00:47.911 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T15:00:47.963 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-16T15:00:47.963 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T15:00:48.016 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T15:00:48.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.249+0000 7f9be9600640 1 Processor -- start 2024-01-16T15:00:48.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.249+0000 7f9be9600640 1 -- start start 2024-01-16T15:00:48.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.250+0000 7f9be9600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9be415e390 0x7f9be4161df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:48.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.250+0000 7f9be9600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9be40ce0b0 con 0x7f9be4157c70 2024-01-16T15:00:48.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.250+0000 7f9be9600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9be4161960 con 0x7f9be40cfb60 2024-01-16T15:00:48.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.250+0000 7f9be9600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9be415e8d0 con 0x7f9be415e390 2024-01-16T15:00:48.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.251+0000 7f9be2a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f9be4157c70 0x7f9be40cf450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50540/0 (socket says 172.21.15.143:50540) 2024-01-16T15:00:48.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.251+0000 7f9be2a00640 1 -- 172.21.15.143:0/1859121855 learned_addr learned my addr 172.21.15.143:0/1859121855 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:48.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.251+0000 7f9be3400640 1 --2- 172.21.15.143:0/1859121855 >> v2:172.21.15.204:3300/0 conn(0x7f9be415e390 0x7f9be4161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:48.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.252+0000 7f9be2000640 1 -- 172.21.15.143:0/1859121855 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2059021445 0 0) 0x7f9be40ce0b0 con 0x7f9be4157c70 2024-01-16T15:00:48.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.252+0000 7f9be2000640 1 -- 172.21.15.143:0/1859121855 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bc4003520 con 0x7f9be4157c70 2024-01-16T15:00:48.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.252+0000 7f9be2000640 1 -- 172.21.15.143:0/1859121855 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 626394113 0 0) 0x7f9bc4003520 con 0x7f9be4157c70 2024-01-16T15:00:48.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.252+0000 7f9be2000640 1 -- 172.21.15.143:0/1859121855 >> v2:172.21.15.204:3300/0 conn(0x7f9be415e390 msgr2=0x7f9be4161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:48.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.252+0000 7f9be2000640 1 --2- 172.21.15.143:0/1859121855 >> v2:172.21.15.204:3300/0 conn(0x7f9be415e390 0x7f9be4161df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:48.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.252+0000 7f9be2000640 1 -- 172.21.15.143:0/1859121855 >> v1:172.21.15.204:6789/0 conn(0x7f9be40cfb60 legacy=0x7f9be415dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:48.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.252+0000 7f9be2000640 1 -- 172.21.15.143:0/1859121855 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be415ebb0 con 0x7f9be4157c70 2024-01-16T15:00:48.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.252+0000 7f9be2000640 1 -- 172.21.15.143:0/1859121855 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7f9bd4001670 con 0x7f9be4157c70 2024-01-16T15:00:48.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.252+0000 7f9be2000640 1 -- 172.21.15.143:0/1859121855 >> v1:172.21.15.143:6789/0 conn(0x7f9be4157c70 legacy=0x7f9be40cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:48.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.252+0000 7f9be2000640 1 --2- 172.21.15.143:0/1859121855 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bc40039c0 0x7f9bc4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:48.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.252+0000 7f9be2000640 1 --2- 172.21.15.143:0/1859121855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bc40047c0 0x7f9bc4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:48.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.252+0000 7f9be2000640 1 --2- 172.21.15.143:0/1859121855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bc40051d0 0x7f9bc400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:48.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.252+0000 7f9be2000640 1 -- 172.21.15.143:0/1859121855 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be415e8d0 con 0x7f9bc40051d0 2024-01-16T15:00:48.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.252+0000 7f9be2000640 1 -- 172.21.15.143:0/1859121855 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc400c120 con 0x7f9bc40047c0 2024-01-16T15:00:48.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.252+0000 7f9be2000640 1 -- 172.21.15.143:0/1859121855 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc400c2e0 con 0x7f9bc40039c0 2024-01-16T15:00:48.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.252+0000 7f9be3400640 1 --2- 172.21.15.143:0/1859121855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bc40051d0 0x7f9bc400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:48.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.253+0000 7f9be3400640 1 -- 172.21.15.143:0/1859121855 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bc40039c0 msgr2=0x7f9bc4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:48.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.253+0000 7f9be3400640 1 --2- 172.21.15.143:0/1859121855 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bc40039c0 0x7f9bc4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:48.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.253+0000 7f9be3400640 1 -- 172.21.15.143:0/1859121855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bc40047c0 msgr2=0x7f9bc4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:48.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.253+0000 7f9be3400640 1 --2- 172.21.15.143:0/1859121855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bc40047c0 0x7f9bc4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:48.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.253+0000 7f9be3400640 1 -- 172.21.15.143:0/1859121855 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc4003520 con 0x7f9bc40051d0 2024-01-16T15:00:48.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.254+0000 7f9be3400640 1 -- 172.21.15.143:0/1859121855 reap_dead start 2024-01-16T15:00:48.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.254+0000 7f9be3400640 1 --2- 172.21.15.143:0/1859121855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bc40051d0 0x7f9bc400b980 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9be415e800 tx=0x7f9bc40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c943ca89de67002 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:48.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.254+0000 7f9be2000640 1 -- 172.21.15.143:0/1859121855 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc02cbb0 con 0x7f9bc40051d0 2024-01-16T15:00:48.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.254+0000 7f9be2000640 1 -- 172.21.15.143:0/1859121855 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bdc00f040 con 0x7f9bc40051d0 2024-01-16T15:00:48.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.254+0000 7f9be2000640 1 -- 172.21.15.143:0/1859121855 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc034440 con 0x7f9bc40051d0 2024-01-16T15:00:48.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.255+0000 7f9be9600640 1 -- 172.21.15.143:0/1859121855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bc40051d0 msgr2=0x7f9bc400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:48.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.255+0000 7f9be9600640 1 --2- 172.21.15.143:0/1859121855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bc40051d0 0x7f9bc400b980 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9be415e800 tx=0x7f9bc40057f0 comp rx=0 tx=0).stop 2024-01-16T15:00:48.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.255+0000 7f9be9600640 1 -- 172.21.15.143:0/1859121855 shutdown_connections 2024-01-16T15:00:48.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.255+0000 7f9be9600640 1 --2- 172.21.15.143:0/1859121855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bc40051d0 0x7f9bc400b980 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:48.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.255+0000 7f9be9600640 1 -- 172.21.15.143:0/1859121855 >> 172.21.15.143:0/1859121855 conn(0x7f9be406a920 msgr2=0x7f9be406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:48.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.255+0000 7f9be9600640 1 -- 172.21.15.143:0/1859121855 shutdown_connections 2024-01-16T15:00:48.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.255+0000 7f9be9600640 1 -- 172.21.15.143:0/1859121855 wait complete. 2024-01-16T15:00:48.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.257+0000 7f9be9600640 1 Processor -- start 2024-01-16T15:00:48.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.257+0000 7f9be9600640 1 -- start start 2024-01-16T15:00:48.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.258+0000 7f9be9600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bc40051d0 0x7f9be41f5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:48.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.258+0000 7f9be9600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9be40cadd0 0x7f9be40cb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:48.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.258+0000 7f9be9600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9be40ad180 0x7f9be40ad600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:48.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.258+0000 7f9be9600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be40cb790 con 0x7f9be40cadd0 2024-01-16T15:00:48.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.258+0000 7f9be9600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be4145d80 con 0x7f9be40ad180 2024-01-16T15:00:48.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.258+0000 7f9be9600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be40ae7e0 con 0x7f9bc40051d0 2024-01-16T15:00:48.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.258+0000 7f9bd9e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9be40cadd0 0x7f9be40cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:48.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.258+0000 7f9bd9e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9be40cadd0 0x7f9be40cb250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59804/0 (socket says 172.21.15.143:59804) 2024-01-16T15:00:48.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.258+0000 7f9bd9e00640 1 -- 172.21.15.143:0/1484017780 learned_addr learned my addr 172.21.15.143:0/1484017780 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:48.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.258+0000 7f9be2a00640 1 --2- 172.21.15.143:0/1484017780 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bc40051d0 0x7f9be41f5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:48.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.259+0000 7f9be3400640 1 --2- 172.21.15.143:0/1484017780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9be40ad180 0x7f9be40ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:48.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.259+0000 7f9bd9e00640 1 -- 172.21.15.143:0/1484017780 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bc40051d0 msgr2=0x7f9be41f5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:48.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.259+0000 7f9bd9e00640 1 --2- 172.21.15.143:0/1484017780 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bc40051d0 0x7f9be41f5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:48.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.259+0000 7f9bd9e00640 1 -- 172.21.15.143:0/1484017780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9be40ad180 msgr2=0x7f9be40ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:48.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.259+0000 7f9bd9e00640 1 --2- 172.21.15.143:0/1484017780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9be40ad180 0x7f9be40ad600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:48.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.259+0000 7f9bd9e00640 1 -- 172.21.15.143:0/1484017780 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be40adf40 con 0x7f9be40cadd0 2024-01-16T15:00:48.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.259+0000 7f9be3400640 1 --2- 172.21.15.143:0/1484017780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9be40ad180 0x7f9be40ad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:48.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.259+0000 7f9bd9e00640 1 --2- 172.21.15.143:0/1484017780 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9be40cadd0 0x7f9be40cb250 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc00c3c0 tx=0x7f9bcc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:48.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.259+0000 7f9bdb400640 1 -- 172.21.15.143:0/1484017780 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc0052d0 con 0x7f9be40cadd0 2024-01-16T15:00:48.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.259+0000 7f9bdb400640 1 -- 172.21.15.143:0/1484017780 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bcc005470 con 0x7f9be40cadd0 2024-01-16T15:00:48.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.259+0000 7f9bdb400640 1 -- 172.21.15.143:0/1484017780 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc007260 con 0x7f9be40cadd0 2024-01-16T15:00:48.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.259+0000 7f9be9600640 1 -- 172.21.15.143:0/1484017780 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9be40ae220 con 0x7f9be40cadd0 2024-01-16T15:00:48.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.259+0000 7f9be9600640 1 -- 172.21.15.143:0/1484017780 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be41f61c0 con 0x7f9be40cadd0 2024-01-16T15:00:48.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.261+0000 7f9be9600640 1 -- 172.21.15.143:0/1484017780 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9be4145ec0 con 0x7f9be40cadd0 2024-01-16T15:00:48.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.261+0000 7f9bdb400640 1 -- 172.21.15.143:0/1484017780 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f9bcc0073c0 con 0x7f9be40cadd0 2024-01-16T15:00:48.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.263+0000 7f9bdb400640 1 --2- 172.21.15.143:0/1484017780 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f9bbc06b2c0 0x7f9bbc06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:48.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.263+0000 7f9bdb400640 1 -- 172.21.15.143:0/1484017780 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f9bcc088eb0 con 0x7f9be40cadd0 2024-01-16T15:00:48.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.263+0000 7f9be2a00640 1 --2- 172.21.15.143:0/1484017780 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f9bbc06b2c0 0x7f9bbc06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:48.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.264+0000 7f9be2a00640 1 --2- 172.21.15.143:0/1484017780 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f9bbc06b2c0 0x7f9bbc06d780 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4001860 tx=0x7f9bd4001690 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:48.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.267+0000 7f9bdb400640 1 -- 172.21.15.143:0/1484017780 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9bcc053d30 con 0x7f9be40cadd0 2024-01-16T15:00:48.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.456+0000 7f9be9600640 1 -- 172.21.15.143:0/1484017780 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9be4067470 con 0x7f9be40cadd0 2024-01-16T15:00:48.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.459+0000 7f9bdb400640 1 -- 172.21.15.143:0/1484017780 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16814 (secure 0 0 0) 0x7f9bcc057350 con 0x7f9be40cadd0 2024-01-16T15:00:48.440 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:00:48.441 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":22,"fsid":"c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0","created":"2024-01-16T14:59:38.532601+0000","modified":"2024-01-16T15:00:45.626415+0000","last_up_change":"2024-01-16T14:59:50.983699+0000","last_in_change":"2024-01-16T14:59:42.056297+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:59:51.727720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T15:00:01.010472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T15:00:02.041228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f4c8891d-ab29-472a-907d-8446f20f7012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6801","nonce":1569293626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6805","nonce":1569293626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6814","nonce":1569293626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6809","nonce":1569293626}]},"public_addr":"172.21.15.143:6801/1569293626","cluster_addr":"172.21.15.143:6805/1569293626","heartbeat_back_addr":"172.21.15.143:6814/1569293626","heartbeat_front_addr":"172.21.15.143:6809/1569293626","state":["exists","up"]},{"osd":1,"uuid":"c4a2788d-8b89-4992-82b2-7f4b9debd509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6804","nonce":1276841168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6808","nonce":1276841168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6813","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6815","nonce":1276841168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6811","nonce":1276841168}]},"public_addr":"172.21.15.143:6804/1276841168","cluster_addr":"172.21.15.143:6808/1276841168","heartbeat_back_addr":"172.21.15.143:6815/1276841168","heartbeat_front_addr":"172.21.15.143:6811/1276841168","state":["exists","up"]},{"osd":2,"uuid":"64645c1d-31b2-4794-b0f2-7ccf8df30393","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6817","nonce":1217670170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6819","nonce":1217670170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6823","nonce":1217670170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6821","nonce":1217670170}]},"public_addr":"172.21.15.143:6817/1217670170","cluster_addr":"172.21.15.143:6819/1217670170","heartbeat_back_addr":"172.21.15.143:6823/1217670170","heartbeat_front_addr":"172.21.15.143:6821/1217670170","state":["exists","up"]},{"osd":3,"uuid":"73674129-a84e-4d31-8247-7d0179ae6df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6825","nonce":3414188012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6827","nonce":3414188012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6831","nonce":3414188012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6829","nonce":3414188012}]},"public_addr":"172.21.15.143:6825/3414188012","cluster_addr":"172.21.15.143:6827/3414188012","heartbeat_back_addr":"172.21.15.143:6831/3414188012","heartbeat_front_addr":"172.21.15.143:6829/3414188012","state":["exists","up"]},{"osd":4,"uuid":"cea28c9b-2f1c-4087-b415-9960789131d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6811","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6812","nonce":1070255716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6816","nonce":1070255716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6823","nonce":1070255716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6820","nonce":1070255716}]},"public_addr":"172.21.15.204:6812/1070255716","cluster_addr":"172.21.15.204:6816/1070255716","heartbeat_back_addr":"172.21.15.204:6823/1070255716","heartbeat_front_addr":"172.21.15.204:6820/1070255716","state":["exists","up"]},{"osd":5,"uuid":"33b63560-5574-4e78-983f-b398d67a2cb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6809","nonce":3605417837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6813","nonce":3605417837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6819","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6821","nonce":3605417837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6817","nonce":3605417837}]},"public_addr":"172.21.15.204:6809/3605417837","cluster_addr":"172.21.15.204:6813/3605417837","heartbeat_back_addr":"172.21.15.204:6821/3605417837","heartbeat_front_addr":"172.21.15.204:6817/3605417837","state":["exists","up"]},{"osd":6,"uuid":"df397427-8940-4eee-9c79-eece669d735d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6801","nonce":1085000534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6803","nonce":1085000534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6808","nonce":1085000534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6805","nonce":1085000534}]},"public_addr":"172.21.15.204:6801/1085000534","cluster_addr":"172.21.15.204:6803/1085000534","heartbeat_back_addr":"172.21.15.204:6808/1085000534","heartbeat_front_addr":"172.21.15.204:6805/1085000534","state":["exists","up"]},{"osd":7,"uuid":"a2e5a15c-8312-405a-ac02-f7e1fc9df2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6825","nonce":1337220609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6827","nonce":1337220609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6831","nonce":1337220609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6829","nonce":1337220609}]},"public_addr":"172.21.15.204:6825/1337220609","cluster_addr":"172.21.15.204:6827/1337220609","heartbeat_back_addr":"172.21.15.204:6831/1337220609","heartbeat_front_addr":"172.21.15.204:6829/1337220609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:49.984507+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.395306+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.632998+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.373627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.342920+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.438366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.331926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.406657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T15:00:48.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.462+0000 7f9be9600640 1 -- 172.21.15.143:0/1484017780 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f9bbc06b2c0 msgr2=0x7f9bbc06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:48.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.462+0000 7f9be9600640 1 --2- 172.21.15.143:0/1484017780 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f9bbc06b2c0 0x7f9bbc06d780 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4001860 tx=0x7f9bd4001690 comp rx=0 tx=0).stop 2024-01-16T15:00:48.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.462+0000 7f9be9600640 1 -- 172.21.15.143:0/1484017780 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9be40cadd0 msgr2=0x7f9be40cb250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:48.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.462+0000 7f9be9600640 1 --2- 172.21.15.143:0/1484017780 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9be40cadd0 0x7f9be40cb250 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc00c3c0 tx=0x7f9bcc00c880 comp rx=0 tx=0).stop 2024-01-16T15:00:48.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.462+0000 7f9be9600640 1 -- 172.21.15.143:0/1484017780 shutdown_connections 2024-01-16T15:00:48.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.462+0000 7f9be9600640 1 --2- 172.21.15.143:0/1484017780 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f9bbc06b2c0 0x7f9bbc06d780 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:48.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.462+0000 7f9be9600640 1 --2- 172.21.15.143:0/1484017780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9be40ad180 0x7f9be40ad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:48.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.462+0000 7f9be9600640 1 --2- 172.21.15.143:0/1484017780 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9be40cadd0 0x7f9be40cb250 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:48.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.462+0000 7f9be9600640 1 --2- 172.21.15.143:0/1484017780 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bc40051d0 0x7f9be41f5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:48.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.462+0000 7f9be9600640 1 -- 172.21.15.143:0/1484017780 >> 172.21.15.143:0/1484017780 conn(0x7f9be406a920 msgr2=0x7f9be40ce400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:48.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.462+0000 7f9be9600640 1 -- 172.21.15.143:0/1484017780 shutdown_connections 2024-01-16T15:00:48.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.462+0000 7f9be9600640 1 -- 172.21.15.143:0/1484017780 wait complete. 2024-01-16T15:00:48.460 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-16T14:59:51.727720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-16T15:00:01.010472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-16T15:00:02.041228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-16T15:00:48.461 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-16T15:00:48.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.696+0000 7f5e8c600640 1 Processor -- start 2024-01-16T15:00:48.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.696+0000 7f5e8c600640 1 -- start start 2024-01-16T15:00:48.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.697+0000 7f5e8c600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5e840ccbd0 0x7f5e841660b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:48.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.697+0000 7f5e8c600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e841655f0 con 0x7f5e840ccfb0 2024-01-16T15:00:48.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.697+0000 7f5e8c600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e840cc380 con 0x7f5e840c9650 2024-01-16T15:00:48.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.697+0000 7f5e8c600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e84162b30 con 0x7f5e840ccbd0 2024-01-16T15:00:48.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.698+0000 7f5e89200640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f5e840ccfb0 0x7f5e84162000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50550/0 (socket says 172.21.15.143:50550) 2024-01-16T15:00:48.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.698+0000 7f5e89200640 1 -- 172.21.15.143:0/1598381721 learned_addr learned my addr 172.21.15.143:0/1598381721 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:48.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.698+0000 7f5e83e00640 1 -- 172.21.15.143:0/1598381721 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3171486964 0 0) 0x7f5e841655f0 con 0x7f5e840ccfb0 2024-01-16T15:00:48.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.698+0000 7f5e83e00640 1 -- 172.21.15.143:0/1598381721 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e64003520 con 0x7f5e840ccfb0 2024-01-16T15:00:48.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.698+0000 7f5e89c00640 1 --2- 172.21.15.143:0/1598381721 >> v2:172.21.15.204:3300/0 conn(0x7f5e840ccbd0 0x7f5e841660b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:48.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.698+0000 7f5e83e00640 1 -- 172.21.15.143:0/1598381721 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3337607946 0 0) 0x7f5e64003520 con 0x7f5e840ccfb0 2024-01-16T15:00:48.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.698+0000 7f5e83e00640 1 -- 172.21.15.143:0/1598381721 >> v2:172.21.15.204:3300/0 conn(0x7f5e840ccbd0 msgr2=0x7f5e841660b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:48.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.698+0000 7f5e83e00640 1 --2- 172.21.15.143:0/1598381721 >> v2:172.21.15.204:3300/0 conn(0x7f5e840ccbd0 0x7f5e841660b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:48.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.698+0000 7f5e83e00640 1 -- 172.21.15.143:0/1598381721 >> v1:172.21.15.204:6789/0 conn(0x7f5e840c9650 legacy=0x7f5e840c9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:48.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.699+0000 7f5e83e00640 1 -- 172.21.15.143:0/1598381721 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e84162e10 con 0x7f5e840ccfb0 2024-01-16T15:00:48.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.699+0000 7f5e83e00640 1 -- 172.21.15.143:0/1598381721 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7f5e6c001670 con 0x7f5e840ccfb0 2024-01-16T15:00:48.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.699+0000 7f5e83e00640 1 -- 172.21.15.143:0/1598381721 >> v1:172.21.15.143:6789/0 conn(0x7f5e840ccfb0 legacy=0x7f5e84162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:48.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.699+0000 7f5e83e00640 1 --2- 172.21.15.143:0/1598381721 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e64003970 0x7f5e64003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:48.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.699+0000 7f5e83e00640 1 --2- 172.21.15.143:0/1598381721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e640047c0 0x7f5e64004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:48.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.699+0000 7f5e83e00640 1 --2- 172.21.15.143:0/1598381721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e64005180 0x7f5e6400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:48.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.699+0000 7f5e83e00640 1 -- 172.21.15.143:0/1598381721 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e84162b30 con 0x7f5e640047c0 2024-01-16T15:00:48.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.699+0000 7f5e83e00640 1 -- 172.21.15.143:0/1598381721 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6400c0d0 con 0x7f5e64005180 2024-01-16T15:00:48.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.699+0000 7f5e83e00640 1 -- 172.21.15.143:0/1598381721 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6400c290 con 0x7f5e64003970 2024-01-16T15:00:48.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.699+0000 7f5e89200640 1 --2- 172.21.15.143:0/1598381721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e640047c0 0x7f5e64004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:48.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.699+0000 7f5e8a600640 1 --2- 172.21.15.143:0/1598381721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e64005180 0x7f5e6400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:48.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.699+0000 7f5e89c00640 1 --2- 172.21.15.143:0/1598381721 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e64003970 0x7f5e64003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:48.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.699+0000 7f5e89200640 1 -- 172.21.15.143:0/1598381721 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e64003970 msgr2=0x7f5e64003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:48.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.699+0000 7f5e89200640 1 --2- 172.21.15.143:0/1598381721 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e64003970 0x7f5e64003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:48.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.699+0000 7f5e89200640 1 -- 172.21.15.143:0/1598381721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e64005180 msgr2=0x7f5e6400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:48.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.699+0000 7f5e89200640 1 --2- 172.21.15.143:0/1598381721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e64005180 0x7f5e6400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:48.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.699+0000 7f5e8a600640 1 -- 172.21.15.143:0/1598381721 reap_dead start 2024-01-16T15:00:48.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.699+0000 7f5e89200640 1 -- 172.21.15.143:0/1598381721 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e64003520 con 0x7f5e640047c0 2024-01-16T15:00:48.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.699+0000 7f5e89c00640 1 --2- 172.21.15.143:0/1598381721 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e64003970 0x7f5e64003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:48.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.701+0000 7f5e89200640 1 --2- 172.21.15.143:0/1598381721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e640047c0 0x7f5e64004c40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c00c240 tx=0x7f5e640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368bbd65c650acbc server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:48.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.701+0000 7f5e83e00640 1 -- 172.21.15.143:0/1598381721 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c005ad0 con 0x7f5e640047c0 2024-01-16T15:00:48.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.701+0000 7f5e83e00640 1 -- 172.21.15.143:0/1598381721 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6c005c50 con 0x7f5e640047c0 2024-01-16T15:00:48.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.701+0000 7f5e83e00640 1 -- 172.21.15.143:0/1598381721 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c02c8b0 con 0x7f5e640047c0 2024-01-16T15:00:48.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.701+0000 7f5e8c600640 1 -- 172.21.15.143:0/1598381721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e640047c0 msgr2=0x7f5e64004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:48.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.701+0000 7f5e8c600640 1 --2- 172.21.15.143:0/1598381721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e640047c0 0x7f5e64004c40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c00c240 tx=0x7f5e640057f0 comp rx=0 tx=0).stop 2024-01-16T15:00:48.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.701+0000 7f5e8c600640 1 -- 172.21.15.143:0/1598381721 shutdown_connections 2024-01-16T15:00:48.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.701+0000 7f5e8c600640 1 --2- 172.21.15.143:0/1598381721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e640047c0 0x7f5e64004c40 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:48.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.702+0000 7f5e8c600640 1 -- 172.21.15.143:0/1598381721 >> 172.21.15.143:0/1598381721 conn(0x7f5e840b0c50 msgr2=0x7f5e840b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:48.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.702+0000 7f5e8c600640 1 -- 172.21.15.143:0/1598381721 shutdown_connections 2024-01-16T15:00:48.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.702+0000 7f5e8c600640 1 -- 172.21.15.143:0/1598381721 wait complete. 2024-01-16T15:00:48.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.703+0000 7f5e8c600640 1 Processor -- start 2024-01-16T15:00:48.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.703+0000 7f5e8c600640 1 -- start start 2024-01-16T15:00:48.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.704+0000 7f5e8c600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e640047c0 0x7f5e840af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:48.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.704+0000 7f5e8c600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e8415ac00 0x7f5e8415b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:48.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.704+0000 7f5e8c600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e84158020 0x7f5e841584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:48.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.704+0000 7f5e8c600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e840ae0b0 con 0x7f5e84158020 2024-01-16T15:00:48.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.704+0000 7f5e8c600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e841589e0 con 0x7f5e640047c0 2024-01-16T15:00:48.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.704+0000 7f5e8c600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e840b1ec0 con 0x7f5e8415ac00 2024-01-16T15:00:48.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.705+0000 7f5e8a600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e84158020 0x7f5e841584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:48.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.705+0000 7f5e8a600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e84158020 0x7f5e841584a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59818/0 (socket says 172.21.15.143:59818) 2024-01-16T15:00:48.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.705+0000 7f5e8a600640 1 -- 172.21.15.143:0/4289855122 learned_addr learned my addr 172.21.15.143:0/4289855122 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:48.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.705+0000 7f5e89c00640 1 --2- 172.21.15.143:0/4289855122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e640047c0 0x7f5e840af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:48.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.705+0000 7f5e8a600640 1 -- 172.21.15.143:0/4289855122 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e8415ac00 msgr2=0x7f5e8415b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:48.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.705+0000 7f5e89200640 1 --2- 172.21.15.143:0/4289855122 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e8415ac00 0x7f5e8415b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:48.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.705+0000 7f5e8a600640 1 --2- 172.21.15.143:0/4289855122 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e8415ac00 0x7f5e8415b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:48.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.705+0000 7f5e8a600640 1 -- 172.21.15.143:0/4289855122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e640047c0 msgr2=0x7f5e840af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:48.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.705+0000 7f5e8a600640 1 --2- 172.21.15.143:0/4289855122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e640047c0 0x7f5e840af6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:48.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.705+0000 7f5e8a600640 1 -- 172.21.15.143:0/4289855122 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e840b21a0 con 0x7f5e84158020 2024-01-16T15:00:48.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.705+0000 7f5e8a600640 1 --2- 172.21.15.143:0/4289855122 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e84158020 0x7f5e841584a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5e8409d280 tx=0x7f5e7c00e670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:48.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.706+0000 7f5e81600640 1 -- 172.21.15.143:0/4289855122 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c0050d0 con 0x7f5e84158020 2024-01-16T15:00:48.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.706+0000 7f5e81600640 1 -- 172.21.15.143:0/4289855122 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e7c005230 con 0x7f5e84158020 2024-01-16T15:00:48.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.706+0000 7f5e81600640 1 -- 172.21.15.143:0/4289855122 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c0035f0 con 0x7f5e84158020 2024-01-16T15:00:48.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.706+0000 7f5e8c600640 1 -- 172.21.15.143:0/4289855122 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e840b2480 con 0x7f5e84158020 2024-01-16T15:00:48.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.706+0000 7f5e8c600640 1 -- 172.21.15.143:0/4289855122 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e840b2a40 con 0x7f5e84158020 2024-01-16T15:00:48.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.707+0000 7f5e8c600640 1 -- 172.21.15.143:0/4289855122 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e84061a50 con 0x7f5e84158020 2024-01-16T15:00:48.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.708+0000 7f5e81600640 1 -- 172.21.15.143:0/4289855122 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f5e7c003750 con 0x7f5e84158020 2024-01-16T15:00:48.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.709+0000 7f5e81600640 1 --2- 172.21.15.143:0/4289855122 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f5e5c06b2c0 0x7f5e5c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:48.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.709+0000 7f5e81600640 1 -- 172.21.15.143:0/4289855122 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f5e7c089230 con 0x7f5e84158020 2024-01-16T15:00:48.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.709+0000 7f5e89c00640 1 --2- 172.21.15.143:0/4289855122 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f5e5c06b2c0 0x7f5e5c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:48.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.710+0000 7f5e89c00640 1 --2- 172.21.15.143:0/4289855122 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f5e5c06b2c0 0x7f5e5c06d780 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5e840681f0 tx=0x7f5e7400b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:48.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.713+0000 7f5e81600640 1 -- 172.21.15.143:0/4289855122 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5e7c053f80 con 0x7f5e84158020 2024-01-16T15:00:48.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.902+0000 7f5e8c600640 1 -- 172.21.15.143:0/4289855122 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5e8404e8b0 con 0x7f5e84158020 2024-01-16T15:00:48.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.903+0000 7f5e81600640 1 -- 172.21.15.143:0/4289855122 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5e7c0575a0 con 0x7f5e84158020 2024-01-16T15:00:48.884 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-16T15:00:48.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.906+0000 7f5e8c600640 1 -- 172.21.15.143:0/4289855122 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f5e5c06b2c0 msgr2=0x7f5e5c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:48.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.906+0000 7f5e8c600640 1 --2- 172.21.15.143:0/4289855122 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f5e5c06b2c0 0x7f5e5c06d780 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5e840681f0 tx=0x7f5e7400b040 comp rx=0 tx=0).stop 2024-01-16T15:00:48.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.906+0000 7f5e8c600640 1 -- 172.21.15.143:0/4289855122 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e84158020 msgr2=0x7f5e841584a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:48.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.906+0000 7f5e8c600640 1 --2- 172.21.15.143:0/4289855122 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e84158020 0x7f5e841584a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5e8409d280 tx=0x7f5e7c00e670 comp rx=0 tx=0).stop 2024-01-16T15:00:48.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.906+0000 7f5e8c600640 1 -- 172.21.15.143:0/4289855122 shutdown_connections 2024-01-16T15:00:48.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.906+0000 7f5e8c600640 1 --2- 172.21.15.143:0/4289855122 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f5e5c06b2c0 0x7f5e5c06d780 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:48.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.906+0000 7f5e8c600640 1 --2- 172.21.15.143:0/4289855122 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e84158020 0x7f5e841584a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:48.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.906+0000 7f5e8c600640 1 --2- 172.21.15.143:0/4289855122 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e8415ac00 0x7f5e8415b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:48.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.906+0000 7f5e8c600640 1 --2- 172.21.15.143:0/4289855122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e640047c0 0x7f5e840af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:48.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.906+0000 7f5e8c600640 1 -- 172.21.15.143:0/4289855122 >> 172.21.15.143:0/4289855122 conn(0x7f5e840b0c50 msgr2=0x7f5e840c57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:48.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.906+0000 7f5e8c600640 1 -- 172.21.15.143:0/4289855122 shutdown_connections 2024-01-16T15:00:48.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:48.906+0000 7f5e8c600640 1 -- 172.21.15.143:0/4289855122 wait complete. 2024-01-16T15:00:48.907 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-16T15:00:49.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.138+0000 7f5406600640 1 Processor -- start 2024-01-16T15:00:49.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.138+0000 7f5406600640 1 -- start start 2024-01-16T15:00:49.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.140+0000 7f5406600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f540015b440 0x7f54000cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:49.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.140+0000 7f5406600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f54000cbf20 0x7f54000c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:49.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.140+0000 7f5406600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f540015eea0 con 0x7f54000c57e0 2024-01-16T15:00:49.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.140+0000 7f5406600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54000cc3d0 con 0x7f54000cbf20 2024-01-16T15:00:49.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.140+0000 7f5406600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f540015b820 con 0x7f540015b440 2024-01-16T15:00:49.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.140+0000 7f53f7e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f54000cbf20 0x7f54000c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:49.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.140+0000 7f53f7e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f54000cbf20 0x7f54000c52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59834/0 (socket says 172.21.15.143:59834) 2024-01-16T15:00:49.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.140+0000 7f53ffe00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f54000c57e0 0x7f54000c3ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50566/0 (socket says 172.21.15.143:50566) 2024-01-16T15:00:49.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.140+0000 7f53f7e00640 1 -- 172.21.15.143:0/152723862 learned_addr learned my addr 172.21.15.143:0/152723862 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:49.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.140+0000 7f53ff400640 1 --2- 172.21.15.143:0/152723862 >> v2:172.21.15.204:3300/0 conn(0x7f540015b440 0x7f54000cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:49.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.141+0000 7f53f7e00640 1 -- 172.21.15.143:0/152723862 >> v2:172.21.15.204:3300/0 conn(0x7f540015b440 msgr2=0x7f54000cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:49.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.141+0000 7f53f7e00640 1 --2- 172.21.15.143:0/152723862 >> v2:172.21.15.204:3300/0 conn(0x7f540015b440 0x7f54000cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:49.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.141+0000 7f53f7e00640 1 -- 172.21.15.143:0/152723862 >> v1:172.21.15.143:6789/0 conn(0x7f54000c57e0 legacy=0x7f54000c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:49.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.141+0000 7f53f7e00640 1 -- 172.21.15.143:0/152723862 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f540006d600 con 0x7f54000cbf20 2024-01-16T15:00:49.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.142+0000 7f53f7e00640 1 --2- 172.21.15.143:0/152723862 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54000cbf20 0x7f54000c52a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f540006e770 tx=0x7f53e802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1401358a66f37cc server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:49.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.142+0000 7f53fea00640 1 -- 172.21.15.143:0/152723862 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e8031870 con 0x7f54000cbf20 2024-01-16T15:00:49.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.142+0000 7f53fea00640 1 -- 172.21.15.143:0/152723862 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e80319d0 con 0x7f54000cbf20 2024-01-16T15:00:49.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.142+0000 7f53fea00640 1 -- 172.21.15.143:0/152723862 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e8031d60 con 0x7f54000cbf20 2024-01-16T15:00:49.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.142+0000 7f5406600640 1 -- 172.21.15.143:0/152723862 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54000cbf20 msgr2=0x7f54000c52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:49.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.142+0000 7f5406600640 1 --2- 172.21.15.143:0/152723862 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54000cbf20 0x7f54000c52a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f540006e770 tx=0x7f53e802b910 comp rx=0 tx=0).stop 2024-01-16T15:00:49.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.143+0000 7f5406600640 1 -- 172.21.15.143:0/152723862 shutdown_connections 2024-01-16T15:00:49.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.143+0000 7f5406600640 1 --2- 172.21.15.143:0/152723862 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54000cbf20 0x7f54000c52a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:49.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.143+0000 7f5406600640 1 --2- 172.21.15.143:0/152723862 >> v2:172.21.15.204:3300/0 conn(0x7f540015b440 0x7f54000cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:49.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.143+0000 7f5406600640 1 -- 172.21.15.143:0/152723862 >> 172.21.15.143:0/152723862 conn(0x7f54000b0c50 msgr2=0x7f54000b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:49.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.143+0000 7f5406600640 1 -- 172.21.15.143:0/152723862 shutdown_connections 2024-01-16T15:00:49.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.143+0000 7f5406600640 1 -- 172.21.15.143:0/152723862 wait complete. 2024-01-16T15:00:49.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.144+0000 7f5406600640 1 Processor -- start 2024-01-16T15:00:49.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.145+0000 7f5406600640 1 -- start start 2024-01-16T15:00:49.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.145+0000 7f5406600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54000c57e0 0x7f5400154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:49.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.146+0000 7f5406600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54000cbf20 0x7f5400099e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:49.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.146+0000 7f5406600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f540015b440 0x7f54000b4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:49.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.146+0000 7f5406600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54000cd050 con 0x7f54000c57e0 2024-01-16T15:00:49.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.146+0000 7f5406600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54000b4bb0 con 0x7f54000cbf20 2024-01-16T15:00:49.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.146+0000 7f5406600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f540009a500 con 0x7f540015b440 2024-01-16T15:00:49.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.146+0000 7f53ff400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54000c57e0 0x7f5400154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:49.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.146+0000 7f53ff400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54000c57e0 0x7f5400154d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59840/0 (socket says 172.21.15.143:59840) 2024-01-16T15:00:49.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.146+0000 7f53ff400640 1 -- 172.21.15.143:0/2427753048 learned_addr learned my addr 172.21.15.143:0/2427753048 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:49.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.146+0000 7f53ff400640 1 -- 172.21.15.143:0/2427753048 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f540015b440 msgr2=0x7f54000b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:49.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.146+0000 7f53ff400640 1 --2- 172.21.15.143:0/2427753048 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f540015b440 0x7f54000b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:49.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.146+0000 7f53ff400640 1 -- 172.21.15.143:0/2427753048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54000cbf20 msgr2=0x7f5400099e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:49.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.146+0000 7f53ff400640 1 --2- 172.21.15.143:0/2427753048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54000cbf20 0x7f5400099e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:49.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.146+0000 7f53ff400640 1 -- 172.21.15.143:0/2427753048 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e80094d0 con 0x7f54000c57e0 2024-01-16T15:00:49.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.146+0000 7f53ff400640 1 --2- 172.21.15.143:0/2427753048 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54000c57e0 0x7f5400154d30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f53f000d620 tx=0x7f53f000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:49.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.147+0000 7f53f7400640 1 -- 172.21.15.143:0/2427753048 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f000b620 con 0x7f54000c57e0 2024-01-16T15:00:49.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.147+0000 7f53f7400640 1 -- 172.21.15.143:0/2427753048 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53f000b780 con 0x7f54000c57e0 2024-01-16T15:00:49.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.147+0000 7f53f7400640 1 -- 172.21.15.143:0/2427753048 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f000bb30 con 0x7f54000c57e0 2024-01-16T15:00:49.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.147+0000 7f5406600640 1 -- 172.21.15.143:0/2427753048 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f540009a7e0 con 0x7f54000c57e0 2024-01-16T15:00:49.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.147+0000 7f5406600640 1 -- 172.21.15.143:0/2427753048 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f540009aac0 con 0x7f54000c57e0 2024-01-16T15:00:49.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.148+0000 7f5406600640 1 -- 172.21.15.143:0/2427753048 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f540004e8b0 con 0x7f54000c57e0 2024-01-16T15:00:49.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.149+0000 7f53f7400640 1 -- 172.21.15.143:0/2427753048 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f53f000bc90 con 0x7f54000c57e0 2024-01-16T15:00:49.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.150+0000 7f53f7400640 1 --2- 172.21.15.143:0/2427753048 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f53d806b2c0 0x7f53d806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:49.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.150+0000 7f53f7400640 1 -- 172.21.15.143:0/2427753048 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f53f008a130 con 0x7f54000c57e0 2024-01-16T15:00:49.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.150+0000 7f53f7e00640 1 --2- 172.21.15.143:0/2427753048 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f53d806b2c0 0x7f53d806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:49.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.151+0000 7f53f7e00640 1 --2- 172.21.15.143:0/2427753048 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f53d806b2c0 0x7f53d806d780 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f53e8036000 tx=0x7f53e8030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:49.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.155+0000 7f53f7400640 1 -- 172.21.15.143:0/2427753048 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f53f0054f10 con 0x7f54000c57e0 2024-01-16T15:00:49.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.345+0000 7f5406600640 1 -- 172.21.15.143:0/2427753048 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f540006dc80 con 0x7f54000c57e0 2024-01-16T15:00:49.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.346+0000 7f53f7400640 1 -- 172.21.15.143:0/2427753048 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f53f0058530 con 0x7f54000c57e0 2024-01-16T15:00:49.327 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2024-01-16T15:00:49.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.348+0000 7f5406600640 1 -- 172.21.15.143:0/2427753048 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f53d806b2c0 msgr2=0x7f53d806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:49.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.348+0000 7f5406600640 1 --2- 172.21.15.143:0/2427753048 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f53d806b2c0 0x7f53d806d780 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f53e8036000 tx=0x7f53e8030000 comp rx=0 tx=0).stop 2024-01-16T15:00:49.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.348+0000 7f5406600640 1 -- 172.21.15.143:0/2427753048 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54000c57e0 msgr2=0x7f5400154d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:49.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.348+0000 7f5406600640 1 --2- 172.21.15.143:0/2427753048 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54000c57e0 0x7f5400154d30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f53f000d620 tx=0x7f53f000dae0 comp rx=0 tx=0).stop 2024-01-16T15:00:49.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.349+0000 7f5406600640 1 -- 172.21.15.143:0/2427753048 shutdown_connections 2024-01-16T15:00:49.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.349+0000 7f5406600640 1 --2- 172.21.15.143:0/2427753048 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f53d806b2c0 0x7f53d806d780 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:49.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.349+0000 7f5406600640 1 --2- 172.21.15.143:0/2427753048 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f540015b440 0x7f54000b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:49.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.349+0000 7f5406600640 1 --2- 172.21.15.143:0/2427753048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54000cbf20 0x7f5400099e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:49.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.349+0000 7f5406600640 1 --2- 172.21.15.143:0/2427753048 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54000c57e0 0x7f5400154d30 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:49.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.349+0000 7f5406600640 1 -- 172.21.15.143:0/2427753048 >> 172.21.15.143:0/2427753048 conn(0x7f54000b0c50 msgr2=0x7f54000c4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:49.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.349+0000 7f5406600640 1 -- 172.21.15.143:0/2427753048 shutdown_connections 2024-01-16T15:00:49.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.349+0000 7f5406600640 1 -- 172.21.15.143:0/2427753048 wait complete. 2024-01-16T15:00:49.347 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-16T15:00:49.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.579+0000 7f5b72400640 1 Processor -- start 2024-01-16T15:00:49.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.579+0000 7f5b72400640 1 -- start start 2024-01-16T15:00:49.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.581+0000 7f5b72400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5b6c0c67c0 0x7f5b6c161400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:49.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.581+0000 7f5b72400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5b6c161940 0x7f5b6c06ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:49.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.581+0000 7f5b72400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b6c15de00 con 0x7f5b6c140520 2024-01-16T15:00:49.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.581+0000 7f5b72400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b6c15e0e0 con 0x7f5b6c0c67c0 2024-01-16T15:00:49.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.581+0000 7f5b72400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b6c15e3c0 con 0x7f5b6c161940 2024-01-16T15:00:49.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.581+0000 7f5b63e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5b6c0c67c0 0x7f5b6c161400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:49.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.581+0000 7f5b63e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5b6c0c67c0 0x7f5b6c161400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59848/0 (socket says 172.21.15.143:59848) 2024-01-16T15:00:49.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.581+0000 7f5b63e00640 1 -- 172.21.15.143:0/656640324 learned_addr learned my addr 172.21.15.143:0/656640324 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:49.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.581+0000 7f5b6be00640 1 --2- 172.21.15.143:0/656640324 >> v2:172.21.15.204:3300/0 conn(0x7f5b6c161940 0x7f5b6c06ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:49.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.582+0000 7f5b63e00640 1 -- 172.21.15.143:0/656640324 >> v2:172.21.15.204:3300/0 conn(0x7f5b6c161940 msgr2=0x7f5b6c06ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:49.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.582+0000 7f5b63e00640 1 --2- 172.21.15.143:0/656640324 >> v2:172.21.15.204:3300/0 conn(0x7f5b6c161940 0x7f5b6c06ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:49.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.582+0000 7f5b63e00640 1 -- 172.21.15.143:0/656640324 >> v1:172.21.15.204:6789/0 conn(0x7f5b6c140520 legacy=0x7f5b6c0c60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:49.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.582+0000 7f5b63e00640 1 -- 172.21.15.143:0/656640324 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b6c15e6a0 con 0x7f5b6c0c67c0 2024-01-16T15:00:49.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.583+0000 7f5b63e00640 1 --2- 172.21.15.143:0/656640324 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b6c0c67c0 0x7f5b6c161400 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c0ceb50 tx=0x7f5b5402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=942ce5cc4e947626 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:49.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.583+0000 7f5b6aa00640 1 -- 172.21.15.143:0/656640324 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b54031870 con 0x7f5b6c0c67c0 2024-01-16T15:00:49.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.583+0000 7f5b6aa00640 1 -- 172.21.15.143:0/656640324 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b540319d0 con 0x7f5b6c0c67c0 2024-01-16T15:00:49.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.583+0000 7f5b6aa00640 1 -- 172.21.15.143:0/656640324 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b54031d60 con 0x7f5b6c0c67c0 2024-01-16T15:00:49.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.584+0000 7f5b72400640 1 -- 172.21.15.143:0/656640324 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b6c0c67c0 msgr2=0x7f5b6c161400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:49.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.584+0000 7f5b72400640 1 --2- 172.21.15.143:0/656640324 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b6c0c67c0 0x7f5b6c161400 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c0ceb50 tx=0x7f5b5402b910 comp rx=0 tx=0).stop 2024-01-16T15:00:49.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.584+0000 7f5b72400640 1 -- 172.21.15.143:0/656640324 shutdown_connections 2024-01-16T15:00:49.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.584+0000 7f5b72400640 1 --2- 172.21.15.143:0/656640324 >> v2:172.21.15.204:3300/0 conn(0x7f5b6c161940 0x7f5b6c06ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:49.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.584+0000 7f5b72400640 1 --2- 172.21.15.143:0/656640324 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b6c0c67c0 0x7f5b6c161400 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:49.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.584+0000 7f5b72400640 1 -- 172.21.15.143:0/656640324 >> 172.21.15.143:0/656640324 conn(0x7f5b6c0b2e40 msgr2=0x7f5b6c0b3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:49.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.584+0000 7f5b72400640 1 -- 172.21.15.143:0/656640324 shutdown_connections 2024-01-16T15:00:49.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.584+0000 7f5b72400640 1 -- 172.21.15.143:0/656640324 wait complete. 2024-01-16T15:00:49.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.587+0000 7f5b72400640 1 Processor -- start 2024-01-16T15:00:49.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.587+0000 7f5b72400640 1 -- start start 2024-01-16T15:00:49.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.588+0000 7f5b72400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b6c0c67c0 0x7f5b6c153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:49.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.588+0000 7f5b72400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b6c140520 0x7f5b6c06c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:49.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.588+0000 7f5b72400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b6c161940 0x7f5b6c06ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:49.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.588+0000 7f5b72400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6c153ad0 con 0x7f5b6c161940 2024-01-16T15:00:49.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.588+0000 7f5b72400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6c158b10 con 0x7f5b6c0c67c0 2024-01-16T15:00:49.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.588+0000 7f5b72400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6c158df0 con 0x7f5b6c140520 2024-01-16T15:00:49.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.588+0000 7f5b6be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b6c161940 0x7f5b6c06ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:49.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.588+0000 7f5b6be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b6c161940 0x7f5b6c06ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59852/0 (socket says 172.21.15.143:59852) 2024-01-16T15:00:49.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.588+0000 7f5b6be00640 1 -- 172.21.15.143:0/3874585349 learned_addr learned my addr 172.21.15.143:0/3874585349 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:49.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.589+0000 7f5b63e00640 1 --2- 172.21.15.143:0/3874585349 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b6c140520 0x7f5b6c06c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:49.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.589+0000 7f5b6b400640 1 --2- 172.21.15.143:0/3874585349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b6c0c67c0 0x7f5b6c153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:49.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.589+0000 7f5b6be00640 1 -- 172.21.15.143:0/3874585349 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b6c140520 msgr2=0x7f5b6c06c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:49.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.589+0000 7f5b6be00640 1 --2- 172.21.15.143:0/3874585349 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b6c140520 0x7f5b6c06c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:49.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.589+0000 7f5b6be00640 1 -- 172.21.15.143:0/3874585349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b6c0c67c0 msgr2=0x7f5b6c153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:49.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.589+0000 7f5b6be00640 1 --2- 172.21.15.143:0/3874585349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b6c0c67c0 0x7f5b6c153590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:49.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.589+0000 7f5b6be00640 1 -- 172.21.15.143:0/3874585349 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b540094d0 con 0x7f5b6c161940 2024-01-16T15:00:49.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.589+0000 7f5b6be00640 1 --2- 172.21.15.143:0/3874585349 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b6c161940 0x7f5b6c06ffe0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5b64007e60 tx=0x7f5b6400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:49.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.589+0000 7f5b63400640 1 -- 172.21.15.143:0/3874585349 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b64014420 con 0x7f5b6c161940 2024-01-16T15:00:49.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.589+0000 7f5b63400640 1 -- 172.21.15.143:0/3874585349 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b640145a0 con 0x7f5b6c161940 2024-01-16T15:00:49.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.590+0000 7f5b72400640 1 -- 172.21.15.143:0/3874585349 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b6c1590d0 con 0x7f5b6c161940 2024-01-16T15:00:49.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.590+0000 7f5b63400640 1 -- 172.21.15.143:0/3874585349 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b64014950 con 0x7f5b6c161940 2024-01-16T15:00:49.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.590+0000 7f5b72400640 1 -- 172.21.15.143:0/3874585349 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b6c158800 con 0x7f5b6c161940 2024-01-16T15:00:49.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.591+0000 7f5b72400640 1 -- 172.21.15.143:0/3874585349 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b6c04e8b0 con 0x7f5b6c161940 2024-01-16T15:00:49.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.592+0000 7f5b63400640 1 -- 172.21.15.143:0/3874585349 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f5b64014ab0 con 0x7f5b6c161940 2024-01-16T15:00:49.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.593+0000 7f5b63400640 1 --2- 172.21.15.143:0/3874585349 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f5b4406b2c0 0x7f5b4406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:49.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.593+0000 7f5b63400640 1 -- 172.21.15.143:0/3874585349 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f5b6408a640 con 0x7f5b6c161940 2024-01-16T15:00:49.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.593+0000 7f5b6b400640 1 --2- 172.21.15.143:0/3874585349 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f5b4406b2c0 0x7f5b4406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:49.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.594+0000 7f5b6b400640 1 --2- 172.21.15.143:0/3874585349 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f5b4406b2c0 0x7f5b4406d780 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c004ea0 tx=0x7f5b5c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:49.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.597+0000 7f5b63400640 1 -- 172.21.15.143:0/3874585349 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5b64055390 con 0x7f5b6c161940 2024-01-16T15:00:49.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.789+0000 7f5b72400640 1 -- 172.21.15.143:0/3874585349 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5b6c159af0 con 0x7f5b6c161940 2024-01-16T15:00:49.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.790+0000 7f5b63400640 1 -- 172.21.15.143:0/3874585349 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5b640589b0 con 0x7f5b6c161940 2024-01-16T15:00:49.771 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2024-01-16T15:00:49.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.793+0000 7f5b72400640 1 -- 172.21.15.143:0/3874585349 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f5b4406b2c0 msgr2=0x7f5b4406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:49.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.793+0000 7f5b72400640 1 --2- 172.21.15.143:0/3874585349 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f5b4406b2c0 0x7f5b4406d780 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c004ea0 tx=0x7f5b5c009040 comp rx=0 tx=0).stop 2024-01-16T15:00:49.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.793+0000 7f5b72400640 1 -- 172.21.15.143:0/3874585349 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b6c161940 msgr2=0x7f5b6c06ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:49.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.793+0000 7f5b72400640 1 --2- 172.21.15.143:0/3874585349 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b6c161940 0x7f5b6c06ffe0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5b64007e60 tx=0x7f5b6400e4b0 comp rx=0 tx=0).stop 2024-01-16T15:00:49.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.793+0000 7f5b72400640 1 -- 172.21.15.143:0/3874585349 shutdown_connections 2024-01-16T15:00:49.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.793+0000 7f5b72400640 1 --2- 172.21.15.143:0/3874585349 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f5b4406b2c0 0x7f5b4406d780 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:49.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.793+0000 7f5b72400640 1 --2- 172.21.15.143:0/3874585349 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b6c161940 0x7f5b6c06ffe0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:49.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.793+0000 7f5b72400640 1 --2- 172.21.15.143:0/3874585349 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b6c140520 0x7f5b6c06c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:49.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.793+0000 7f5b72400640 1 --2- 172.21.15.143:0/3874585349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b6c0c67c0 0x7f5b6c153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:49.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.793+0000 7f5b72400640 1 -- 172.21.15.143:0/3874585349 >> 172.21.15.143:0/3874585349 conn(0x7f5b6c0b2e40 msgr2=0x7f5b6c15a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:49.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.793+0000 7f5b72400640 1 -- 172.21.15.143:0/3874585349 shutdown_connections 2024-01-16T15:00:49.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:49.793+0000 7f5b72400640 1 -- 172.21.15.143:0/3874585349 wait complete. 2024-01-16T15:00:49.793 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-16T15:00:49.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.009+0000 7f644a400640 1 Processor -- start 2024-01-16T15:00:49.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.009+0000 7f644a400640 1 -- start start 2024-01-16T15:00:49.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.011+0000 7f644a400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f644406cf70 0x7f64440cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:49.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.011+0000 7f644a400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f644406d350 0x7f64440ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:49.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.011+0000 7f644a400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64440c4990 con 0x7f64440cc200 2024-01-16T15:00:49.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.011+0000 7f644a400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64440cc7c0 con 0x7f644406cf70 2024-01-16T15:00:49.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.011+0000 7f644a400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64440c5e30 con 0x7f644406d350 2024-01-16T15:00:49.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.011+0000 7f6443400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f644406cf70 0x7f64440cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:49.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.011+0000 7f6443400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f644406cf70 0x7f64440cbcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59866/0 (socket says 172.21.15.143:59866) 2024-01-16T15:00:49.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.011+0000 7f6443400640 1 -- 172.21.15.143:0/300601252 learned_addr learned my addr 172.21.15.143:0/300601252 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:49.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.011+0000 7f6443400640 1 -- 172.21.15.143:0/300601252 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f644406d350 msgr2=0x7f64440ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:49.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.011+0000 7f6443400640 1 --2- 172.21.15.143:0/300601252 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f644406d350 0x7f64440ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:49.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.011+0000 7f6443400640 1 -- 172.21.15.143:0/300601252 >> v1:172.21.15.204:6789/0 conn(0x7f64440cc200 legacy=0x7f64440ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:49.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.011+0000 7f6443400640 1 -- 172.21.15.143:0/300601252 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64440c4060 con 0x7f644406cf70 2024-01-16T15:00:49.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.012+0000 7f6443400640 1 --2- 172.21.15.143:0/300601252 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f644406cf70 0x7f64440cbcc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6434009870 tx=0x7f643402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8820248ccc00cd49 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:49.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.013+0000 7f6442a00640 1 -- 172.21.15.143:0/300601252 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6434031870 con 0x7f644406cf70 2024-01-16T15:00:49.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.013+0000 7f6442a00640 1 -- 172.21.15.143:0/300601252 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64340319d0 con 0x7f644406cf70 2024-01-16T15:00:49.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.013+0000 7f6442a00640 1 -- 172.21.15.143:0/300601252 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6434031d60 con 0x7f644406cf70 2024-01-16T15:00:49.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.013+0000 7f644a400640 1 -- 172.21.15.143:0/300601252 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f644406cf70 msgr2=0x7f64440cbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:49.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.013+0000 7f644a400640 1 --2- 172.21.15.143:0/300601252 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f644406cf70 0x7f64440cbcc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6434009870 tx=0x7f643402b910 comp rx=0 tx=0).stop 2024-01-16T15:00:49.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.013+0000 7f644a400640 1 -- 172.21.15.143:0/300601252 shutdown_connections 2024-01-16T15:00:49.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.013+0000 7f644a400640 1 --2- 172.21.15.143:0/300601252 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f644406d350 0x7f64440ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:49.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.013+0000 7f644a400640 1 --2- 172.21.15.143:0/300601252 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f644406cf70 0x7f64440cbcc0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:49.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.013+0000 7f644a400640 1 -- 172.21.15.143:0/300601252 >> 172.21.15.143:0/300601252 conn(0x7f64440b0c50 msgr2=0x7f64440b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:49.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.014+0000 7f644a400640 1 -- 172.21.15.143:0/300601252 shutdown_connections 2024-01-16T15:00:49.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.014+0000 7f644a400640 1 -- 172.21.15.143:0/300601252 wait complete. 2024-01-16T15:00:49.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.015+0000 7f644a400640 1 Processor -- start 2024-01-16T15:00:49.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.015+0000 7f644a400640 1 -- start start 2024-01-16T15:00:49.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.016+0000 7f644a400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f644406cf70 0x7f64440c6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:49.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.016+0000 7f644a400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f644406d350 0x7f64440adda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:49.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.016+0000 7f644a400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64440cc200 0x7f644415f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:49.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.016+0000 7f644a400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64440c94a0 con 0x7f64440cc200 2024-01-16T15:00:49.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.016+0000 7f644a400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644415f860 con 0x7f644406cf70 2024-01-16T15:00:49.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.016+0000 7f644a400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6444099b70 con 0x7f644406d350 2024-01-16T15:00:49.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.017+0000 7f6443e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64440cc200 0x7f644415f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:49.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.017+0000 7f6443e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64440cc200 0x7f644415f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59880/0 (socket says 172.21.15.143:59880) 2024-01-16T15:00:49.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.017+0000 7f6443e00640 1 -- 172.21.15.143:0/1150214897 learned_addr learned my addr 172.21.15.143:0/1150214897 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:49.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.017+0000 7f643be00640 1 --2- 172.21.15.143:0/1150214897 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f644406d350 0x7f64440adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:50.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.017+0000 7f6443e00640 1 -- 172.21.15.143:0/1150214897 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f644406d350 msgr2=0x7f64440adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:50.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.017+0000 7f6443e00640 1 --2- 172.21.15.143:0/1150214897 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f644406d350 0x7f64440adda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:50.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.017+0000 7f6443e00640 1 -- 172.21.15.143:0/1150214897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f644406cf70 msgr2=0x7f64440c6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:50.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.017+0000 7f6443e00640 1 --2- 172.21.15.143:0/1150214897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f644406cf70 0x7f64440c6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:50.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.017+0000 7f6443e00640 1 -- 172.21.15.143:0/1150214897 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64340094d0 con 0x7f64440cc200 2024-01-16T15:00:50.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.017+0000 7f6443e00640 1 --2- 172.21.15.143:0/1150214897 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64440cc200 0x7f644415f320 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f643c00b990 tx=0x7f643c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:50.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.018+0000 7f643b400640 1 -- 172.21.15.143:0/1150214897 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643c00cba0 con 0x7f64440cc200 2024-01-16T15:00:50.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.018+0000 7f643b400640 1 -- 172.21.15.143:0/1150214897 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f643c00cd00 con 0x7f64440cc200 2024-01-16T15:00:50.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.018+0000 7f643b400640 1 -- 172.21.15.143:0/1150214897 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643c014680 con 0x7f64440cc200 2024-01-16T15:00:50.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.018+0000 7f644a400640 1 -- 172.21.15.143:0/1150214897 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6444099e50 con 0x7f64440cc200 2024-01-16T15:00:50.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.018+0000 7f644a400640 1 -- 172.21.15.143:0/1150214897 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f644409a410 con 0x7f64440cc200 2024-01-16T15:00:50.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.019+0000 7f644a400640 1 -- 172.21.15.143:0/1150214897 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f644409a630 con 0x7f64440cc200 2024-01-16T15:00:50.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.020+0000 7f643b400640 1 -- 172.21.15.143:0/1150214897 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f643c0147e0 con 0x7f64440cc200 2024-01-16T15:00:50.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.021+0000 7f643b400640 1 --2- 172.21.15.143:0/1150214897 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f641c06b2c0 0x7f641c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:50.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.021+0000 7f643b400640 1 -- 172.21.15.143:0/1150214897 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f643c089cc0 con 0x7f64440cc200 2024-01-16T15:00:50.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.021+0000 7f6443400640 1 --2- 172.21.15.143:0/1150214897 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f641c06b2c0 0x7f641c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:50.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.022+0000 7f6443400640 1 --2- 172.21.15.143:0/1150214897 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f641c06b2c0 0x7f641c06d780 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6434036000 tx=0x7f6434030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:50.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.025+0000 7f643b400640 1 -- 172.21.15.143:0/1150214897 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f643c090080 con 0x7f64440cc200 2024-01-16T15:00:50.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.231+0000 7f644a400640 1 -- 172.21.15.143:0/1150214897 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f644404e8b0 con 0x7f64440cc200 2024-01-16T15:00:50.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.232+0000 7f643b400640 1 -- 172.21.15.143:0/1150214897 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f643c019070 con 0x7f64440cc200 2024-01-16T15:00:50.213 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2024-01-16T15:00:50.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.235+0000 7f644a400640 1 -- 172.21.15.143:0/1150214897 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f641c06b2c0 msgr2=0x7f641c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:50.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.235+0000 7f644a400640 1 --2- 172.21.15.143:0/1150214897 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f641c06b2c0 0x7f641c06d780 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6434036000 tx=0x7f6434030000 comp rx=0 tx=0).stop 2024-01-16T15:00:50.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.235+0000 7f644a400640 1 -- 172.21.15.143:0/1150214897 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64440cc200 msgr2=0x7f644415f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:50.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.235+0000 7f644a400640 1 --2- 172.21.15.143:0/1150214897 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64440cc200 0x7f644415f320 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f643c00b990 tx=0x7f643c00be50 comp rx=0 tx=0).stop 2024-01-16T15:00:50.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.235+0000 7f644a400640 1 -- 172.21.15.143:0/1150214897 shutdown_connections 2024-01-16T15:00:50.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.235+0000 7f644a400640 1 --2- 172.21.15.143:0/1150214897 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f641c06b2c0 0x7f641c06d780 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:50.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.235+0000 7f644a400640 1 --2- 172.21.15.143:0/1150214897 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64440cc200 0x7f644415f320 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:50.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.235+0000 7f644a400640 1 --2- 172.21.15.143:0/1150214897 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f644406d350 0x7f64440adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:50.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.235+0000 7f644a400640 1 --2- 172.21.15.143:0/1150214897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f644406cf70 0x7f64440c6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:50.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.235+0000 7f644a400640 1 -- 172.21.15.143:0/1150214897 >> 172.21.15.143:0/1150214897 conn(0x7f64440b0c50 msgr2=0x7f64440cb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:50.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.236+0000 7f644a400640 1 -- 172.21.15.143:0/1150214897 shutdown_connections 2024-01-16T15:00:50.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.236+0000 7f644a400640 1 -- 172.21.15.143:0/1150214897 wait complete. 2024-01-16T15:00:50.235 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-01-16T15:00:50.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:50.236 DEBUG:teuthology.orchestra.run.smithi204:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-01-16T15:00:50.341 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi204.front.sepia.ceph.com 2024-01-16T15:00:50.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:50.342 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-16T15:00:50.404 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T15:00:50.404 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T15:00:50.461 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T15:00:50.461 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T15:00:50.512 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T15:00:50.512 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T15:00:50.566 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T15:00:50.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.780+0000 7f90d9a00640 1 Processor -- start 2024-01-16T15:00:50.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.780+0000 7f90d9a00640 1 -- start start 2024-01-16T15:00:50.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.781+0000 7f90d9a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f90d40b2250 0x7f90d415df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:50.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.782+0000 7f90d9a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90d415ab80 con 0x7f90d40b2630 2024-01-16T15:00:50.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.782+0000 7f90d9a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90d415e680 con 0x7f90d40b2aa0 2024-01-16T15:00:50.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.782+0000 7f90d9a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90d415a990 con 0x7f90d40b2250 2024-01-16T15:00:50.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.782+0000 7f90cbe00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f90d40b2630 0x7f90d40ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50574/0 (socket says 172.21.15.143:50574) 2024-01-16T15:00:50.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.782+0000 7f90cbe00640 1 -- 172.21.15.143:0/1069389428 learned_addr learned my addr 172.21.15.143:0/1069389428 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:50.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.782+0000 7f90d2a00640 1 --2- 172.21.15.143:0/1069389428 >> v2:172.21.15.204:3300/0 conn(0x7f90d40b2250 0x7f90d415df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:50.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.782+0000 7f90d2000640 1 -- 172.21.15.143:0/1069389428 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1826884288 0 0) 0x7f90d415ab80 con 0x7f90d40b2630 2024-01-16T15:00:50.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.783+0000 7f90d2000640 1 -- 172.21.15.143:0/1069389428 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90b4003520 con 0x7f90d40b2630 2024-01-16T15:00:50.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.783+0000 7f90d2000640 1 -- 172.21.15.143:0/1069389428 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2597880729 0 0) 0x7f90b4003520 con 0x7f90d40b2630 2024-01-16T15:00:50.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.783+0000 7f90d2000640 1 -- 172.21.15.143:0/1069389428 >> v2:172.21.15.204:3300/0 conn(0x7f90d40b2250 msgr2=0x7f90d415df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:50.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.783+0000 7f90d2000640 1 --2- 172.21.15.143:0/1069389428 >> v2:172.21.15.204:3300/0 conn(0x7f90d40b2250 0x7f90d415df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:50.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.783+0000 7f90d2000640 1 -- 172.21.15.143:0/1069389428 >> v1:172.21.15.204:6789/0 conn(0x7f90d40b2aa0 legacy=0x7f90d415b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:50.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.783+0000 7f90d2000640 1 -- 172.21.15.143:0/1069389428 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d415e960 con 0x7f90d40b2630 2024-01-16T15:00:50.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.783+0000 7f90d2000640 1 -- 172.21.15.143:0/1069389428 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7f90bc001670 con 0x7f90d40b2630 2024-01-16T15:00:50.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.783+0000 7f90d2000640 1 -- 172.21.15.143:0/1069389428 >> v1:172.21.15.143:6789/0 conn(0x7f90d40b2630 legacy=0x7f90d40ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:50.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.783+0000 7f90d2000640 1 --2- 172.21.15.143:0/1069389428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90b4003970 0x7f90b4003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:50.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.783+0000 7f90d2000640 1 --2- 172.21.15.143:0/1069389428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90b40047c0 0x7f90b4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:50.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.783+0000 7f90d2000640 1 --2- 172.21.15.143:0/1069389428 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90b4005180 0x7f90b400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:50.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.783+0000 7f90d2000640 1 -- 172.21.15.143:0/1069389428 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b400c050 con 0x7f90b4003970 2024-01-16T15:00:50.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.783+0000 7f90d2000640 1 -- 172.21.15.143:0/1069389428 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b400c190 con 0x7f90b40047c0 2024-01-16T15:00:50.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.783+0000 7f90d2000640 1 -- 172.21.15.143:0/1069389428 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90b400c350 con 0x7f90b4005180 2024-01-16T15:00:50.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.783+0000 7f90d2a00640 1 --2- 172.21.15.143:0/1069389428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90b4003970 0x7f90b4003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:50.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.784+0000 7f90cbe00640 1 --2- 172.21.15.143:0/1069389428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90b40047c0 0x7f90b4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:50.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.784+0000 7f90d2a00640 1 -- 172.21.15.143:0/1069389428 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90b4005180 msgr2=0x7f90b400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:50.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.784+0000 7f90d3400640 1 --2- 172.21.15.143:0/1069389428 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90b4005180 0x7f90b400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:50.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.784+0000 7f90d2a00640 1 --2- 172.21.15.143:0/1069389428 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90b4005180 0x7f90b400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:50.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.784+0000 7f90d2a00640 1 -- 172.21.15.143:0/1069389428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90b40047c0 msgr2=0x7f90b4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:50.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.784+0000 7f90d2a00640 1 --2- 172.21.15.143:0/1069389428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90b40047c0 0x7f90b4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:50.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.784+0000 7f90d2a00640 1 -- 172.21.15.143:0/1069389428 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b4003520 con 0x7f90b4003970 2024-01-16T15:00:50.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.784+0000 7f90d3400640 1 --2- 172.21.15.143:0/1069389428 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90b4005180 0x7f90b400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:50.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.784+0000 7f90cbe00640 1 --2- 172.21.15.143:0/1069389428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90b40047c0 0x7f90b4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:50.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.784+0000 7f90d3400640 1 -- 172.21.15.143:0/1069389428 reap_dead start 2024-01-16T15:00:50.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.785+0000 7f90d2a00640 1 --2- 172.21.15.143:0/1069389428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90b4003970 0x7f90b4003db0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f90c4009ac0 tx=0x7f90b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=633e08d5ee757c41 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:50.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.785+0000 7f90d2000640 1 -- 172.21.15.143:0/1069389428 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c402c6b0 con 0x7f90b4003970 2024-01-16T15:00:50.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.785+0000 7f90d2000640 1 -- 172.21.15.143:0/1069389428 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c402c810 con 0x7f90b4003970 2024-01-16T15:00:50.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.785+0000 7f90d2000640 1 -- 172.21.15.143:0/1069389428 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c402cba0 con 0x7f90b4003970 2024-01-16T15:00:50.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.786+0000 7f90d9a00640 1 -- 172.21.15.143:0/1069389428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90b4003970 msgr2=0x7f90b4003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:50.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.786+0000 7f90d9a00640 1 --2- 172.21.15.143:0/1069389428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90b4003970 0x7f90b4003db0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f90c4009ac0 tx=0x7f90b40057f0 comp rx=0 tx=0).stop 2024-01-16T15:00:50.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.786+0000 7f90d9a00640 1 -- 172.21.15.143:0/1069389428 shutdown_connections 2024-01-16T15:00:50.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.786+0000 7f90d9a00640 1 --2- 172.21.15.143:0/1069389428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90b4003970 0x7f90b4003db0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:50.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.786+0000 7f90d9a00640 1 -- 172.21.15.143:0/1069389428 >> 172.21.15.143:0/1069389428 conn(0x7f90d406a920 msgr2=0x7f90d406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:50.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.786+0000 7f90d9a00640 1 -- 172.21.15.143:0/1069389428 shutdown_connections 2024-01-16T15:00:50.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.786+0000 7f90d9a00640 1 -- 172.21.15.143:0/1069389428 wait complete. 2024-01-16T15:00:50.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.788+0000 7f90d9a00640 1 Processor -- start 2024-01-16T15:00:50.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.788+0000 7f90d9a00640 1 -- start start 2024-01-16T15:00:50.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.789+0000 7f90d9a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90b4003970 0x7f90d4070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:50.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.789+0000 7f90d9a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90d4153ea0 0x7f90d40c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:50.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.789+0000 7f90d9a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90d415cf90 0x7f90d415d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:50.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.789+0000 7f90d9a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d4070b40 con 0x7f90d415cf90 2024-01-16T15:00:50.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.789+0000 7f90d9a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d40b1960 con 0x7f90b4003970 2024-01-16T15:00:50.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.789+0000 7f90d9a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d415d950 con 0x7f90d4153ea0 2024-01-16T15:00:50.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.789+0000 7f90d3400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90d415cf90 0x7f90d415d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:50.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.789+0000 7f90d3400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90d415cf90 0x7f90d415d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59898/0 (socket says 172.21.15.143:59898) 2024-01-16T15:00:50.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.789+0000 7f90d3400640 1 -- 172.21.15.143:0/3315648571 learned_addr learned my addr 172.21.15.143:0/3315648571 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:50.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.790+0000 7f90d2a00640 1 --2- 172.21.15.143:0/3315648571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90b4003970 0x7f90d4070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:50.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.790+0000 7f90d3400640 1 -- 172.21.15.143:0/3315648571 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90d4153ea0 msgr2=0x7f90d40c8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:50.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.790+0000 7f90d3400640 1 --2- 172.21.15.143:0/3315648571 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90d4153ea0 0x7f90d40c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:50.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.790+0000 7f90d3400640 1 -- 172.21.15.143:0/3315648571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90b4003970 msgr2=0x7f90d4070600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:50.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.790+0000 7f90d3400640 1 --2- 172.21.15.143:0/3315648571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90b4003970 0x7f90d4070600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:50.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.790+0000 7f90d3400640 1 -- 172.21.15.143:0/3315648571 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d41f76d0 con 0x7f90d415cf90 2024-01-16T15:00:50.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.790+0000 7f90d3400640 1 --2- 172.21.15.143:0/3315648571 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90d415cf90 0x7f90d415d410 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f90d40d0040 tx=0x7f90cc00d900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:50.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.790+0000 7f90caa00640 1 -- 172.21.15.143:0/3315648571 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90cc00b070 con 0x7f90d415cf90 2024-01-16T15:00:50.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.790+0000 7f90caa00640 1 -- 172.21.15.143:0/3315648571 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90cc003da0 con 0x7f90d415cf90 2024-01-16T15:00:50.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.790+0000 7f90caa00640 1 -- 172.21.15.143:0/3315648571 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90cc005310 con 0x7f90d415cf90 2024-01-16T15:00:50.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.790+0000 7f90d9a00640 1 -- 172.21.15.143:0/3315648571 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90d41f6ac0 con 0x7f90d415cf90 2024-01-16T15:00:50.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.790+0000 7f90d9a00640 1 -- 172.21.15.143:0/3315648571 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90d41f7080 con 0x7f90d415cf90 2024-01-16T15:00:50.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.791+0000 7f90d9a00640 1 -- 172.21.15.143:0/3315648571 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90d4067470 con 0x7f90d415cf90 2024-01-16T15:00:50.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.793+0000 7f90caa00640 1 -- 172.21.15.143:0/3315648571 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f90cc005470 con 0x7f90d415cf90 2024-01-16T15:00:50.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.793+0000 7f90caa00640 1 --2- 172.21.15.143:0/3315648571 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f90ac06b2c0 0x7f90ac06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:50.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.793+0000 7f90caa00640 1 -- 172.21.15.143:0/3315648571 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f90cc089260 con 0x7f90d415cf90 2024-01-16T15:00:50.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.794+0000 7f90d2a00640 1 --2- 172.21.15.143:0/3315648571 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f90ac06b2c0 0x7f90ac06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:50.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.794+0000 7f90d2a00640 1 --2- 172.21.15.143:0/3315648571 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f90ac06b2c0 0x7f90ac06d780 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f90c4009ac0 tx=0x7f90c4030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:50.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.798+0000 7f90caa00640 1 -- 172.21.15.143:0/3315648571 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f90cc053fb0 con 0x7f90d415cf90 2024-01-16T15:00:50.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.988+0000 7f90d9a00640 1 -- 172.21.15.143:0/3315648571 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90d41f76d0 con 0x7f90d415cf90 2024-01-16T15:00:50.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.990+0000 7f90caa00640 1 -- 172.21.15.143:0/3315648571 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16814 (secure 0 0 0) 0x7f90d41f76d0 con 0x7f90d415cf90 2024-01-16T15:00:50.972 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:00:50.972 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":22,"fsid":"c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0","created":"2024-01-16T14:59:38.532601+0000","modified":"2024-01-16T15:00:45.626415+0000","last_up_change":"2024-01-16T14:59:50.983699+0000","last_in_change":"2024-01-16T14:59:42.056297+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:59:51.727720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T15:00:01.010472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T15:00:02.041228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f4c8891d-ab29-472a-907d-8446f20f7012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6801","nonce":1569293626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6805","nonce":1569293626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6814","nonce":1569293626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6809","nonce":1569293626}]},"public_addr":"172.21.15.143:6801/1569293626","cluster_addr":"172.21.15.143:6805/1569293626","heartbeat_back_addr":"172.21.15.143:6814/1569293626","heartbeat_front_addr":"172.21.15.143:6809/1569293626","state":["exists","up"]},{"osd":1,"uuid":"c4a2788d-8b89-4992-82b2-7f4b9debd509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6804","nonce":1276841168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6808","nonce":1276841168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6813","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6815","nonce":1276841168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6811","nonce":1276841168}]},"public_addr":"172.21.15.143:6804/1276841168","cluster_addr":"172.21.15.143:6808/1276841168","heartbeat_back_addr":"172.21.15.143:6815/1276841168","heartbeat_front_addr":"172.21.15.143:6811/1276841168","state":["exists","up"]},{"osd":2,"uuid":"64645c1d-31b2-4794-b0f2-7ccf8df30393","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6817","nonce":1217670170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6819","nonce":1217670170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6823","nonce":1217670170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6821","nonce":1217670170}]},"public_addr":"172.21.15.143:6817/1217670170","cluster_addr":"172.21.15.143:6819/1217670170","heartbeat_back_addr":"172.21.15.143:6823/1217670170","heartbeat_front_addr":"172.21.15.143:6821/1217670170","state":["exists","up"]},{"osd":3,"uuid":"73674129-a84e-4d31-8247-7d0179ae6df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6825","nonce":3414188012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6827","nonce":3414188012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6831","nonce":3414188012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6829","nonce":3414188012}]},"public_addr":"172.21.15.143:6825/3414188012","cluster_addr":"172.21.15.143:6827/3414188012","heartbeat_back_addr":"172.21.15.143:6831/3414188012","heartbeat_front_addr":"172.21.15.143:6829/3414188012","state":["exists","up"]},{"osd":4,"uuid":"cea28c9b-2f1c-4087-b415-9960789131d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6811","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6812","nonce":1070255716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6816","nonce":1070255716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6823","nonce":1070255716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6820","nonce":1070255716}]},"public_addr":"172.21.15.204:6812/1070255716","cluster_addr":"172.21.15.204:6816/1070255716","heartbeat_back_addr":"172.21.15.204:6823/1070255716","heartbeat_front_addr":"172.21.15.204:6820/1070255716","state":["exists","up"]},{"osd":5,"uuid":"33b63560-5574-4e78-983f-b398d67a2cb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6809","nonce":3605417837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6813","nonce":3605417837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6819","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6821","nonce":3605417837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6817","nonce":3605417837}]},"public_addr":"172.21.15.204:6809/3605417837","cluster_addr":"172.21.15.204:6813/3605417837","heartbeat_back_addr":"172.21.15.204:6821/3605417837","heartbeat_front_addr":"172.21.15.204:6817/3605417837","state":["exists","up"]},{"osd":6,"uuid":"df397427-8940-4eee-9c79-eece669d735d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6801","nonce":1085000534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6803","nonce":1085000534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6808","nonce":1085000534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6805","nonce":1085000534}]},"public_addr":"172.21.15.204:6801/1085000534","cluster_addr":"172.21.15.204:6803/1085000534","heartbeat_back_addr":"172.21.15.204:6808/1085000534","heartbeat_front_addr":"172.21.15.204:6805/1085000534","state":["exists","up"]},{"osd":7,"uuid":"a2e5a15c-8312-405a-ac02-f7e1fc9df2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6825","nonce":1337220609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6827","nonce":1337220609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6831","nonce":1337220609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6829","nonce":1337220609}]},"public_addr":"172.21.15.204:6825/1337220609","cluster_addr":"172.21.15.204:6827/1337220609","heartbeat_back_addr":"172.21.15.204:6831/1337220609","heartbeat_front_addr":"172.21.15.204:6829/1337220609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:49.984507+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.395306+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.632998+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.373627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.342920+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.438366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.331926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.406657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T15:00:50.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.993+0000 7f90d9a00640 1 -- 172.21.15.143:0/3315648571 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f90ac06b2c0 msgr2=0x7f90ac06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:50.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.993+0000 7f90d9a00640 1 --2- 172.21.15.143:0/3315648571 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f90ac06b2c0 0x7f90ac06d780 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f90c4009ac0 tx=0x7f90c4030000 comp rx=0 tx=0).stop 2024-01-16T15:00:50.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.993+0000 7f90d9a00640 1 -- 172.21.15.143:0/3315648571 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90d415cf90 msgr2=0x7f90d415d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:50.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.993+0000 7f90d9a00640 1 --2- 172.21.15.143:0/3315648571 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90d415cf90 0x7f90d415d410 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f90d40d0040 tx=0x7f90cc00d900 comp rx=0 tx=0).stop 2024-01-16T15:00:50.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.993+0000 7f90d9a00640 1 -- 172.21.15.143:0/3315648571 shutdown_connections 2024-01-16T15:00:50.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.993+0000 7f90d9a00640 1 --2- 172.21.15.143:0/3315648571 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f90ac06b2c0 0x7f90ac06d780 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:50.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.993+0000 7f90d9a00640 1 --2- 172.21.15.143:0/3315648571 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90d415cf90 0x7f90d415d410 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:50.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.993+0000 7f90d9a00640 1 --2- 172.21.15.143:0/3315648571 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90d4153ea0 0x7f90d40c8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:50.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.993+0000 7f90d9a00640 1 --2- 172.21.15.143:0/3315648571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90b4003970 0x7f90d4070600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:50.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.993+0000 7f90d9a00640 1 -- 172.21.15.143:0/3315648571 >> 172.21.15.143:0/3315648571 conn(0x7f90d406a920 msgr2=0x7f90d40cf420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:50.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.994+0000 7f90d9a00640 1 -- 172.21.15.143:0/3315648571 shutdown_connections 2024-01-16T15:00:50.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:50.994+0000 7f90d9a00640 1 -- 172.21.15.143:0/3315648571 wait complete. 2024-01-16T15:00:50.993 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-16T14:59:51.727720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-16T15:00:01.010472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-16T15:00:02.041228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-16T15:00:50.994 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-16T15:00:51.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.229+0000 7fbd38400640 1 Processor -- start 2024-01-16T15:00:51.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.229+0000 7fbd38400640 1 -- start start 2024-01-16T15:00:51.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.230+0000 7fbd38400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbd3015b440 0x7fbd300cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:51.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.230+0000 7fbd38400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd300cbf20 0x7fbd300c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:51.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.230+0000 7fbd38400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd3015eea0 con 0x7fbd300c57e0 2024-01-16T15:00:51.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.231+0000 7fbd38400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd300cc3d0 con 0x7fbd3015b440 2024-01-16T15:00:51.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.231+0000 7fbd38400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3015b820 con 0x7fbd300cbf20 2024-01-16T15:00:51.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.231+0000 7fbd35a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbd3015b440 0x7fbd300cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:51.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.231+0000 7fbd35a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbd3015b440 0x7fbd300cb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:45274/0 (socket says 172.21.15.143:45274) 2024-01-16T15:00:51.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.231+0000 7fbd35a00640 1 -- 172.21.15.143:0/859098322 learned_addr learned my addr 172.21.15.143:0/859098322 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:51.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.231+0000 7fbd35000640 1 --2- 172.21.15.143:0/859098322 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd300cbf20 0x7fbd300c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:51.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.231+0000 7fbd36400640 1 --1- 172.21.15.143:0/859098322 >> v1:172.21.15.204:6789/0 conn(0x7fbd300c57e0 0x7fbd300c3ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:40468/0 (socket says 172.21.15.143:40468) 2024-01-16T15:00:51.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.232+0000 7fbd35a00640 1 -- 172.21.15.143:0/859098322 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd300cbf20 msgr2=0x7fbd300c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:51.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.232+0000 7fbd35a00640 1 --2- 172.21.15.143:0/859098322 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd300cbf20 0x7fbd300c52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:51.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.232+0000 7fbd35a00640 1 -- 172.21.15.143:0/859098322 >> v1:172.21.15.204:6789/0 conn(0x7fbd300c57e0 legacy=0x7fbd300c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:51.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.232+0000 7fbd35a00640 1 -- 172.21.15.143:0/859098322 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd3006d600 con 0x7fbd3015b440 2024-01-16T15:00:51.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.233+0000 7fbd35a00640 1 --2- 172.21.15.143:0/859098322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd3015b440 0x7fbd300cb9e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbd20009e50 tx=0x7fbd2002bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d043de92ee9d86b server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:51.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.234+0000 7fbd2fe00640 1 -- 172.21.15.143:0/859098322 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2002ca80 con 0x7fbd3015b440 2024-01-16T15:00:51.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.234+0000 7fbd2fe00640 1 -- 172.21.15.143:0/859098322 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd2002cbe0 con 0x7fbd3015b440 2024-01-16T15:00:51.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.234+0000 7fbd2fe00640 1 -- 172.21.15.143:0/859098322 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd200343f0 con 0x7fbd3015b440 2024-01-16T15:00:51.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.234+0000 7fbd38400640 1 -- 172.21.15.143:0/859098322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd3015b440 msgr2=0x7fbd300cb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:51.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.234+0000 7fbd38400640 1 --2- 172.21.15.143:0/859098322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd3015b440 0x7fbd300cb9e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbd20009e50 tx=0x7fbd2002bee0 comp rx=0 tx=0).stop 2024-01-16T15:00:51.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.235+0000 7fbd38400640 1 -- 172.21.15.143:0/859098322 shutdown_connections 2024-01-16T15:00:51.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.235+0000 7fbd38400640 1 --2- 172.21.15.143:0/859098322 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd300cbf20 0x7fbd300c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:51.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.235+0000 7fbd38400640 1 --2- 172.21.15.143:0/859098322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd3015b440 0x7fbd300cb9e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:51.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.235+0000 7fbd38400640 1 -- 172.21.15.143:0/859098322 >> 172.21.15.143:0/859098322 conn(0x7fbd300b0c50 msgr2=0x7fbd300b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:51.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.235+0000 7fbd38400640 1 -- 172.21.15.143:0/859098322 shutdown_connections 2024-01-16T15:00:51.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.235+0000 7fbd38400640 1 -- 172.21.15.143:0/859098322 wait complete. 2024-01-16T15:00:51.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.237+0000 7fbd38400640 1 Processor -- start 2024-01-16T15:00:51.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.237+0000 7fbd38400640 1 -- start start 2024-01-16T15:00:51.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.238+0000 7fbd38400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd300c57e0 0x7fbd300c74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:51.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.238+0000 7fbd38400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd300cbf20 0x7fbd3015d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:51.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.238+0000 7fbd38400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd3015b440 0x7fbd301e6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:51.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.238+0000 7fbd38400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd300cda20 con 0x7fbd300cbf20 2024-01-16T15:00:51.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.238+0000 7fbd38400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3015caa0 con 0x7fbd300c57e0 2024-01-16T15:00:51.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.238+0000 7fbd38400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3015cd80 con 0x7fbd3015b440 2024-01-16T15:00:51.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.239+0000 7fbd35000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd300cbf20 0x7fbd3015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:51.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.239+0000 7fbd35000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd300cbf20 0x7fbd3015d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59914/0 (socket says 172.21.15.143:59914) 2024-01-16T15:00:51.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.239+0000 7fbd35000640 1 -- 172.21.15.143:0/1795230150 learned_addr learned my addr 172.21.15.143:0/1795230150 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:51.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.239+0000 7fbd36400640 1 --2- 172.21.15.143:0/1795230150 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd3015b440 0x7fbd301e6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:51.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.239+0000 7fbd35a00640 1 --2- 172.21.15.143:0/1795230150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd300c57e0 0x7fbd300c74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:51.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.239+0000 7fbd35000640 1 -- 172.21.15.143:0/1795230150 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd3015b440 msgr2=0x7fbd301e6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:51.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.239+0000 7fbd35000640 1 --2- 172.21.15.143:0/1795230150 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd3015b440 0x7fbd301e6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:51.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.239+0000 7fbd35000640 1 -- 172.21.15.143:0/1795230150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd300c57e0 msgr2=0x7fbd300c74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:51.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.239+0000 7fbd35000640 1 --2- 172.21.15.143:0/1795230150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd300c57e0 0x7fbd300c74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:51.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.239+0000 7fbd35000640 1 -- 172.21.15.143:0/1795230150 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd20009b30 con 0x7fbd300cbf20 2024-01-16T15:00:51.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.239+0000 7fbd35a00640 1 --2- 172.21.15.143:0/1795230150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd300c57e0 0x7fbd300c74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:51.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.239+0000 7fbd35000640 1 --2- 172.21.15.143:0/1795230150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd300cbf20 0x7fbd3015d860 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbd1800e9b0 tx=0x7fbd1800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:51.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.240+0000 7fbd2d600640 1 -- 172.21.15.143:0/1795230150 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd18009670 con 0x7fbd300cbf20 2024-01-16T15:00:51.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.240+0000 7fbd2d600640 1 -- 172.21.15.143:0/1795230150 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd1800c9a0 con 0x7fbd300cbf20 2024-01-16T15:00:51.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.240+0000 7fbd2d600640 1 -- 172.21.15.143:0/1795230150 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1800cd50 con 0x7fbd300cbf20 2024-01-16T15:00:51.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.240+0000 7fbd38400640 1 -- 172.21.15.143:0/1795230150 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd3015d060 con 0x7fbd300cbf20 2024-01-16T15:00:51.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.240+0000 7fbd38400640 1 -- 172.21.15.143:0/1795230150 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd300c79f0 con 0x7fbd300cbf20 2024-01-16T15:00:51.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.241+0000 7fbd38400640 1 -- 172.21.15.143:0/1795230150 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd3004e8b0 con 0x7fbd300cbf20 2024-01-16T15:00:51.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.242+0000 7fbd2d600640 1 -- 172.21.15.143:0/1795230150 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fbd18013470 con 0x7fbd300cbf20 2024-01-16T15:00:51.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.243+0000 7fbd2d600640 1 --2- 172.21.15.143:0/1795230150 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbd0806b2c0 0x7fbd0806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:51.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.243+0000 7fbd2d600640 1 -- 172.21.15.143:0/1795230150 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fbd18089f40 con 0x7fbd300cbf20 2024-01-16T15:00:51.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.243+0000 7fbd35a00640 1 --2- 172.21.15.143:0/1795230150 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbd0806b2c0 0x7fbd0806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:51.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.244+0000 7fbd35a00640 1 --2- 172.21.15.143:0/1795230150 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbd0806b2c0 0x7fbd0806d780 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbd20035000 tx=0x7fbd20030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:51.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.247+0000 7fbd2d600640 1 -- 172.21.15.143:0/1795230150 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbd18054c90 con 0x7fbd300cbf20 2024-01-16T15:00:51.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.435+0000 7fbd38400640 1 -- 172.21.15.143:0/1795230150 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbd3006dc80 con 0x7fbd300cbf20 2024-01-16T15:00:51.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.436+0000 7fbd2d600640 1 -- 172.21.15.143:0/1795230150 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbd180170a0 con 0x7fbd300cbf20 2024-01-16T15:00:51.416 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-16T15:00:51.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.438+0000 7fbd38400640 1 -- 172.21.15.143:0/1795230150 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbd0806b2c0 msgr2=0x7fbd0806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:51.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.438+0000 7fbd38400640 1 --2- 172.21.15.143:0/1795230150 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbd0806b2c0 0x7fbd0806d780 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbd20035000 tx=0x7fbd20030000 comp rx=0 tx=0).stop 2024-01-16T15:00:51.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.438+0000 7fbd38400640 1 -- 172.21.15.143:0/1795230150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd300cbf20 msgr2=0x7fbd3015d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:51.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.438+0000 7fbd38400640 1 --2- 172.21.15.143:0/1795230150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd300cbf20 0x7fbd3015d860 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbd1800e9b0 tx=0x7fbd1800ee70 comp rx=0 tx=0).stop 2024-01-16T15:00:51.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.438+0000 7fbd38400640 1 -- 172.21.15.143:0/1795230150 shutdown_connections 2024-01-16T15:00:51.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.438+0000 7fbd38400640 1 --2- 172.21.15.143:0/1795230150 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbd0806b2c0 0x7fbd0806d780 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:51.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.438+0000 7fbd38400640 1 --2- 172.21.15.143:0/1795230150 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd3015b440 0x7fbd301e6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:51.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.438+0000 7fbd38400640 1 --2- 172.21.15.143:0/1795230150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd300cbf20 0x7fbd3015d860 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:51.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.438+0000 7fbd38400640 1 --2- 172.21.15.143:0/1795230150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd300c57e0 0x7fbd300c74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:51.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.438+0000 7fbd38400640 1 -- 172.21.15.143:0/1795230150 >> 172.21.15.143:0/1795230150 conn(0x7fbd300b0c50 msgr2=0x7fbd300ac780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:51.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.438+0000 7fbd38400640 1 -- 172.21.15.143:0/1795230150 shutdown_connections 2024-01-16T15:00:51.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.438+0000 7fbd38400640 1 -- 172.21.15.143:0/1795230150 wait complete. 2024-01-16T15:00:51.438 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-16T15:00:51.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.659+0000 7ff181a00640 1 Processor -- start 2024-01-16T15:00:51.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.660+0000 7ff181a00640 1 -- start start 2024-01-16T15:00:51.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.661+0000 7ff181a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff17c0c59b0 0x7ff17c166230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:51.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.661+0000 7ff181a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff17c0c5dc0 con 0x7ff17c15b440 2024-01-16T15:00:51.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.661+0000 7ff181a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff17c15eea0 con 0x7ff17c0cc0f0 2024-01-16T15:00:51.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.661+0000 7ff181a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff17c15b820 con 0x7ff17c0c59b0 2024-01-16T15:00:51.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.661+0000 7ff17b400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff17c0c59b0 0x7ff17c166230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:51.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.661+0000 7ff17aa00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff17c15b440 0x7ff17c0cb9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50588/0 (socket says 172.21.15.143:50588) 2024-01-16T15:00:51.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.662+0000 7ff17aa00640 1 -- 172.21.15.143:0/1970190482 learned_addr learned my addr 172.21.15.143:0/1970190482 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:51.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.662+0000 7ff179600640 1 -- 172.21.15.143:0/1970190482 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1897941400 0 0) 0x7ff17c0c5dc0 con 0x7ff17c15b440 2024-01-16T15:00:51.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.662+0000 7ff179600640 1 -- 172.21.15.143:0/1970190482 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff15c003520 con 0x7ff17c15b440 2024-01-16T15:00:51.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.662+0000 7ff17b400640 1 -- 172.21.15.143:0/1970190482 >> v1:172.21.15.204:6789/0 conn(0x7ff17c0cc0f0 legacy=0x7ff17c0c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:51.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.662+0000 7ff17b400640 1 -- 172.21.15.143:0/1970190482 >> v1:172.21.15.143:6789/0 conn(0x7ff17c15b440 legacy=0x7ff17c0cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:51.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.662+0000 7ff17b400640 1 -- 172.21.15.143:0/1970190482 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff17c06d600 con 0x7ff17c0c59b0 2024-01-16T15:00:51.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.663+0000 7ff17b400640 1 --2- 172.21.15.143:0/1970190482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff17c0c59b0 0x7ff17c166230 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff174009d90 tx=0x7ff17402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d97a04234d66cc54 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:51.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.663+0000 7ff179600640 1 -- 172.21.15.143:0/1970190482 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17402cc60 con 0x7ff17c0c59b0 2024-01-16T15:00:51.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.663+0000 7ff179600640 1 -- 172.21.15.143:0/1970190482 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff17402cdc0 con 0x7ff17c0c59b0 2024-01-16T15:00:51.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.663+0000 7ff179600640 1 -- 172.21.15.143:0/1970190482 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff174034660 con 0x7ff17c0c59b0 2024-01-16T15:00:51.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.664+0000 7ff181a00640 1 -- 172.21.15.143:0/1970190482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff17c0c59b0 msgr2=0x7ff17c166230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:51.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.664+0000 7ff181a00640 1 --2- 172.21.15.143:0/1970190482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff17c0c59b0 0x7ff17c166230 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff174009d90 tx=0x7ff17402beb0 comp rx=0 tx=0).stop 2024-01-16T15:00:51.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.664+0000 7ff181a00640 1 -- 172.21.15.143:0/1970190482 shutdown_connections 2024-01-16T15:00:51.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.664+0000 7ff181a00640 1 --2- 172.21.15.143:0/1970190482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff17c0c59b0 0x7ff17c166230 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:51.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.664+0000 7ff181a00640 1 -- 172.21.15.143:0/1970190482 >> 172.21.15.143:0/1970190482 conn(0x7ff17c0b0c50 msgr2=0x7ff17c0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:51.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.664+0000 7ff181a00640 1 -- 172.21.15.143:0/1970190482 shutdown_connections 2024-01-16T15:00:51.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.664+0000 7ff181a00640 1 -- 172.21.15.143:0/1970190482 wait complete. 2024-01-16T15:00:51.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.666+0000 7ff181a00640 1 Processor -- start 2024-01-16T15:00:51.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.666+0000 7ff181a00640 1 -- start start 2024-01-16T15:00:51.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.667+0000 7ff181a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff17c0c59b0 0x7ff17c1538b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:51.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.667+0000 7ff181a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff17c0cc0f0 0x7ff17c15a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:51.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.667+0000 7ff181a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff17c15b440 0x7ff17c0ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:51.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.667+0000 7ff181a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c1f49c0 con 0x7ff17c15b440 2024-01-16T15:00:51.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.667+0000 7ff181a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c1f3db0 con 0x7ff17c0cc0f0 2024-01-16T15:00:51.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.667+0000 7ff181a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c1f4090 con 0x7ff17c0c59b0 2024-01-16T15:00:51.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.667+0000 7ff17b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff17c15b440 0x7ff17c0ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:51.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.667+0000 7ff17b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff17c15b440 0x7ff17c0ad1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59932/0 (socket says 172.21.15.143:59932) 2024-01-16T15:00:51.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.667+0000 7ff17b400640 1 -- 172.21.15.143:0/914800485 learned_addr learned my addr 172.21.15.143:0/914800485 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:51.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.668+0000 7ff17aa00640 1 --2- 172.21.15.143:0/914800485 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff17c0c59b0 0x7ff17c1538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:51.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.668+0000 7ff17a000640 1 --2- 172.21.15.143:0/914800485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff17c0cc0f0 0x7ff17c15a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:51.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.668+0000 7ff17b400640 1 -- 172.21.15.143:0/914800485 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff17c0c59b0 msgr2=0x7ff17c1538b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:51.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.668+0000 7ff17b400640 1 --2- 172.21.15.143:0/914800485 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff17c0c59b0 0x7ff17c1538b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:51.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.668+0000 7ff17b400640 1 -- 172.21.15.143:0/914800485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff17c0cc0f0 msgr2=0x7ff17c15a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:51.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.668+0000 7ff17b400640 1 --2- 172.21.15.143:0/914800485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff17c0cc0f0 0x7ff17c15a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:51.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.668+0000 7ff17b400640 1 -- 172.21.15.143:0/914800485 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff174009a70 con 0x7ff17c15b440 2024-01-16T15:00:51.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.668+0000 7ff17b400640 1 --2- 172.21.15.143:0/914800485 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff17c15b440 0x7ff17c0ad1a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff174009d90 tx=0x7ff174004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:51.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.668+0000 7ff172a00640 1 -- 172.21.15.143:0/914800485 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff174040650 con 0x7ff17c15b440 2024-01-16T15:00:51.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.668+0000 7ff172a00640 1 -- 172.21.15.143:0/914800485 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1740407b0 con 0x7ff17c15b440 2024-01-16T15:00:51.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.668+0000 7ff181a00640 1 -- 172.21.15.143:0/914800485 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff17c1f4370 con 0x7ff17c15b440 2024-01-16T15:00:51.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.668+0000 7ff172a00640 1 -- 172.21.15.143:0/914800485 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff174040b60 con 0x7ff17c15b440 2024-01-16T15:00:51.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.669+0000 7ff181a00640 1 -- 172.21.15.143:0/914800485 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff17c15ab90 con 0x7ff17c15b440 2024-01-16T15:00:51.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.670+0000 7ff181a00640 1 -- 172.21.15.143:0/914800485 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff17c1554b0 con 0x7ff17c15b440 2024-01-16T15:00:51.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.670+0000 7ff172a00640 1 -- 172.21.15.143:0/914800485 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7ff174040cc0 con 0x7ff17c15b440 2024-01-16T15:00:51.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.671+0000 7ff172a00640 1 --2- 172.21.15.143:0/914800485 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7ff15406b2c0 0x7ff15406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:51.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.671+0000 7ff172a00640 1 -- 172.21.15.143:0/914800485 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7ff1740aca50 con 0x7ff17c15b440 2024-01-16T15:00:51.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.672+0000 7ff17aa00640 1 --2- 172.21.15.143:0/914800485 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7ff15406b2c0 0x7ff15406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:51.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.672+0000 7ff17aa00640 1 --2- 172.21.15.143:0/914800485 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7ff15406b2c0 0x7ff15406d780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff16c0011e0 tx=0x7ff16c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:51.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.676+0000 7ff172a00640 1 -- 172.21.15.143:0/914800485 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff17c1554b0 con 0x7ff17c15b440 2024-01-16T15:00:51.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.870+0000 7ff181a00640 1 -- 172.21.15.143:0/914800485 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff17c1554b0 con 0x7ff17c15b440 2024-01-16T15:00:51.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.871+0000 7ff172a00640 1 -- 172.21.15.143:0/914800485 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff17c1554b0 con 0x7ff17c15b440 2024-01-16T15:00:51.852 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2024-01-16T15:00:51.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.874+0000 7ff181a00640 1 -- 172.21.15.143:0/914800485 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7ff15406b2c0 msgr2=0x7ff15406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:51.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.874+0000 7ff181a00640 1 --2- 172.21.15.143:0/914800485 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7ff15406b2c0 0x7ff15406d780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff16c0011e0 tx=0x7ff16c009040 comp rx=0 tx=0).stop 2024-01-16T15:00:51.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.874+0000 7ff181a00640 1 -- 172.21.15.143:0/914800485 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff17c15b440 msgr2=0x7ff17c0ad1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:51.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.874+0000 7ff181a00640 1 --2- 172.21.15.143:0/914800485 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff17c15b440 0x7ff17c0ad1a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff174009d90 tx=0x7ff174004a40 comp rx=0 tx=0).stop 2024-01-16T15:00:51.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.874+0000 7ff181a00640 1 -- 172.21.15.143:0/914800485 shutdown_connections 2024-01-16T15:00:51.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.874+0000 7ff181a00640 1 --2- 172.21.15.143:0/914800485 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7ff15406b2c0 0x7ff15406d780 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:51.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.874+0000 7ff181a00640 1 --2- 172.21.15.143:0/914800485 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff17c15b440 0x7ff17c0ad1a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:51.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.874+0000 7ff181a00640 1 --2- 172.21.15.143:0/914800485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff17c0cc0f0 0x7ff17c15a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:51.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.874+0000 7ff181a00640 1 --2- 172.21.15.143:0/914800485 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff17c0c59b0 0x7ff17c1538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:51.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.874+0000 7ff181a00640 1 -- 172.21.15.143:0/914800485 >> 172.21.15.143:0/914800485 conn(0x7ff17c0b0c50 msgr2=0x7ff17c06cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:51.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.874+0000 7ff181a00640 1 -- 172.21.15.143:0/914800485 shutdown_connections 2024-01-16T15:00:51.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:51.874+0000 7ff181a00640 1 -- 172.21.15.143:0/914800485 wait complete. 2024-01-16T15:00:51.872 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-16T15:00:52.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.094+0000 7f6f9be00640 1 Processor -- start 2024-01-16T15:00:52.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.094+0000 7f6f9be00640 1 -- start start 2024-01-16T15:00:52.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.095+0000 7f6f9be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f94161060 0x7f6f94161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:52.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.095+0000 7f6f9be00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6f940b3330 0x7f6f9415da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:52.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.096+0000 7f6f9be00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f9406dcf0 con 0x7f6f940b3790 2024-01-16T15:00:52.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.096+0000 7f6f9be00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f9406ec10 con 0x7f6f940b3330 2024-01-16T15:00:52.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.096+0000 7f6f9be00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f94161980 con 0x7f6f94161060 2024-01-16T15:00:52.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.096+0000 7f6f99400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f94161060 0x7f6f94161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:52.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.096+0000 7f6f98a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6f940b3330 0x7f6f9415da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:52.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.096+0000 7f6f99400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f94161060 0x7f6f94161440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:39644/0 (socket says 172.21.15.143:39644) 2024-01-16T15:00:52.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.096+0000 7f6f99400640 1 -- 172.21.15.143:0/2545883059 learned_addr learned my addr 172.21.15.143:0/2545883059 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:52.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.096+0000 7f6f99e00640 1 --1- 172.21.15.143:0/2545883059 >> v1:172.21.15.204:6789/0 conn(0x7f6f940b3790 0x7f6f9415dfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:40488/0 (socket says 172.21.15.143:40488) 2024-01-16T15:00:52.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.097+0000 7f6f99400640 1 -- 172.21.15.143:0/2545883059 >> v2:172.21.15.204:3300/0 conn(0x7f6f940b3330 msgr2=0x7f6f9415da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:52.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.097+0000 7f6f99400640 1 --2- 172.21.15.143:0/2545883059 >> v2:172.21.15.204:3300/0 conn(0x7f6f940b3330 0x7f6f9415da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:52.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.097+0000 7f6f99400640 1 -- 172.21.15.143:0/2545883059 >> v1:172.21.15.204:6789/0 conn(0x7f6f940b3790 legacy=0x7f6f9415dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:52.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.097+0000 7f6f99400640 1 -- 172.21.15.143:0/2545883059 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f94161ec0 con 0x7f6f94161060 2024-01-16T15:00:52.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.097+0000 7f6f98a00640 1 --2- 172.21.15.143:0/2545883059 >> v2:172.21.15.204:3300/0 conn(0x7f6f940b3330 0x7f6f9415da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T15:00:52.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.099+0000 7f6f99400640 1 --2- 172.21.15.143:0/2545883059 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f94161060 0x7f6f94161440 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6f84009870 tx=0x7f6f8402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74840b15bfd9361f server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:52.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.099+0000 7f6f93e00640 1 -- 172.21.15.143:0/2545883059 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f84031640 con 0x7f6f94161060 2024-01-16T15:00:52.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.099+0000 7f6f93e00640 1 -- 172.21.15.143:0/2545883059 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f840317a0 con 0x7f6f94161060 2024-01-16T15:00:52.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.099+0000 7f6f93e00640 1 -- 172.21.15.143:0/2545883059 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f84031b30 con 0x7f6f94161060 2024-01-16T15:00:52.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.100+0000 7f6f9be00640 1 -- 172.21.15.143:0/2545883059 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f94161060 msgr2=0x7f6f94161440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:52.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.100+0000 7f6f9be00640 1 --2- 172.21.15.143:0/2545883059 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f94161060 0x7f6f94161440 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6f84009870 tx=0x7f6f8402b8f0 comp rx=0 tx=0).stop 2024-01-16T15:00:52.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.100+0000 7f6f9be00640 1 -- 172.21.15.143:0/2545883059 shutdown_connections 2024-01-16T15:00:52.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.100+0000 7f6f9be00640 1 --2- 172.21.15.143:0/2545883059 >> v2:172.21.15.204:3300/0 conn(0x7f6f940b3330 0x7f6f9415da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:52.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.100+0000 7f6f9be00640 1 --2- 172.21.15.143:0/2545883059 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f94161060 0x7f6f94161440 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:52.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.100+0000 7f6f9be00640 1 -- 172.21.15.143:0/2545883059 >> 172.21.15.143:0/2545883059 conn(0x7f6f940cf900 msgr2=0x7f6f940cfd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:52.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.100+0000 7f6f9be00640 1 -- 172.21.15.143:0/2545883059 shutdown_connections 2024-01-16T15:00:52.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.100+0000 7f6f9be00640 1 -- 172.21.15.143:0/2545883059 wait complete. 2024-01-16T15:00:52.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.102+0000 7f6f9be00640 1 Processor -- start 2024-01-16T15:00:52.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.103+0000 7f6f9be00640 1 -- start start 2024-01-16T15:00:52.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.104+0000 7f6f9be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f940b3330 0x7f6f9414d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:52.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.104+0000 7f6f9be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f940b3790 0x7f6f941f3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:52.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.104+0000 7f6f9be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f94161060 0x7f6f94154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:52.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.104+0000 7f6f9be00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f94155660 con 0x7f6f940b3330 2024-01-16T15:00:52.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.104+0000 7f6f9be00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9414de90 con 0x7f6f940b3790 2024-01-16T15:00:52.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.104+0000 7f6f9be00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f941f3bb0 con 0x7f6f94161060 2024-01-16T15:00:52.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.104+0000 7f6f99400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f940b3330 0x7f6f9414d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:52.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.104+0000 7f6f99400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f940b3330 0x7f6f9414d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59942/0 (socket says 172.21.15.143:59942) 2024-01-16T15:00:52.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.104+0000 7f6f99400640 1 -- 172.21.15.143:0/1717462561 learned_addr learned my addr 172.21.15.143:0/1717462561 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:52.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.104+0000 7f6f98a00640 1 --2- 172.21.15.143:0/1717462561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f940b3790 0x7f6f941f3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:52.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.104+0000 7f6f99e00640 1 --2- 172.21.15.143:0/1717462561 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f94161060 0x7f6f94154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:52.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.104+0000 7f6f99400640 1 -- 172.21.15.143:0/1717462561 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f94161060 msgr2=0x7f6f94154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:52.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.104+0000 7f6f99400640 1 --2- 172.21.15.143:0/1717462561 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f94161060 0x7f6f94154e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:52.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.104+0000 7f6f99400640 1 -- 172.21.15.143:0/1717462561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f940b3790 msgr2=0x7f6f941f3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:52.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.104+0000 7f6f99400640 1 --2- 172.21.15.143:0/1717462561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f940b3790 0x7f6f941f3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:52.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.104+0000 7f6f99400640 1 -- 172.21.15.143:0/1717462561 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f7c0093a0 con 0x7f6f940b3330 2024-01-16T15:00:52.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.104+0000 7f6f98a00640 1 --2- 172.21.15.143:0/1717462561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f940b3790 0x7f6f941f3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:52.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.104+0000 7f6f99e00640 1 --2- 172.21.15.143:0/1717462561 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f94161060 0x7f6f94154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:52.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.105+0000 7f6f99400640 1 --2- 172.21.15.143:0/1717462561 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f940b3330 0x7f6f9414d950 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6f84000c00 tx=0x7f6f840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:52.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.105+0000 7f6f91600640 1 -- 172.21.15.143:0/1717462561 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8403a930 con 0x7f6f940b3330 2024-01-16T15:00:52.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.105+0000 7f6f91600640 1 -- 172.21.15.143:0/1717462561 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f8403aa90 con 0x7f6f940b3330 2024-01-16T15:00:52.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.105+0000 7f6f91600640 1 -- 172.21.15.143:0/1717462561 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8403ae40 con 0x7f6f940b3330 2024-01-16T15:00:52.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.105+0000 7f6f9be00640 1 -- 172.21.15.143:0/1717462561 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f840094d0 con 0x7f6f940b3330 2024-01-16T15:00:52.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.105+0000 7f6f9be00640 1 -- 172.21.15.143:0/1717462561 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f941553a0 con 0x7f6f940b3330 2024-01-16T15:00:52.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.107+0000 7f6f9be00640 1 -- 172.21.15.143:0/1717462561 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f9404e8b0 con 0x7f6f940b3330 2024-01-16T15:00:52.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.107+0000 7f6f91600640 1 -- 172.21.15.143:0/1717462561 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f6f8402f050 con 0x7f6f940b3330 2024-01-16T15:00:52.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.108+0000 7f6f91600640 1 --2- 172.21.15.143:0/1717462561 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6f6c06b2f0 0x7f6f6c06d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:52.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.108+0000 7f6f91600640 1 -- 172.21.15.143:0/1717462561 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f6f840ab8f0 con 0x7f6f940b3330 2024-01-16T15:00:52.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.109+0000 7f6f98a00640 1 --2- 172.21.15.143:0/1717462561 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6f6c06b2f0 0x7f6f6c06d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:52.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.109+0000 7f6f98a00640 1 --2- 172.21.15.143:0/1717462561 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6f6c06b2f0 0x7f6f6c06d7b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c009ba0 tx=0x7f6f7c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:52.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.113+0000 7f6f91600640 1 -- 172.21.15.143:0/1717462561 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6f84076770 con 0x7f6f940b3330 2024-01-16T15:00:52.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.302+0000 7f6f9be00640 1 -- 172.21.15.143:0/1717462561 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6f94061490 con 0x7f6f940b3330 2024-01-16T15:00:52.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.303+0000 7f6f91600640 1 -- 172.21.15.143:0/1717462561 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6f840350a0 con 0x7f6f940b3330 2024-01-16T15:00:52.284 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2024-01-16T15:00:52.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.306+0000 7f6f9be00640 1 -- 172.21.15.143:0/1717462561 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6f6c06b2f0 msgr2=0x7f6f6c06d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:52.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.306+0000 7f6f9be00640 1 --2- 172.21.15.143:0/1717462561 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6f6c06b2f0 0x7f6f6c06d7b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c009ba0 tx=0x7f6f7c008040 comp rx=0 tx=0).stop 2024-01-16T15:00:52.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.306+0000 7f6f9be00640 1 -- 172.21.15.143:0/1717462561 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f940b3330 msgr2=0x7f6f9414d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:52.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.306+0000 7f6f9be00640 1 --2- 172.21.15.143:0/1717462561 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f940b3330 0x7f6f9414d950 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6f84000c00 tx=0x7f6f840046c0 comp rx=0 tx=0).stop 2024-01-16T15:00:52.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.306+0000 7f6f9be00640 1 -- 172.21.15.143:0/1717462561 shutdown_connections 2024-01-16T15:00:52.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.306+0000 7f6f9be00640 1 --2- 172.21.15.143:0/1717462561 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6f6c06b2f0 0x7f6f6c06d7b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:52.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.306+0000 7f6f9be00640 1 --2- 172.21.15.143:0/1717462561 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f94161060 0x7f6f94154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:52.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.306+0000 7f6f9be00640 1 --2- 172.21.15.143:0/1717462561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f940b3790 0x7f6f941f3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:52.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.306+0000 7f6f9be00640 1 --2- 172.21.15.143:0/1717462561 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f940b3330 0x7f6f9414d950 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:52.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.306+0000 7f6f9be00640 1 -- 172.21.15.143:0/1717462561 >> 172.21.15.143:0/1717462561 conn(0x7f6f940cf900 msgr2=0x7f6f940cfce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:52.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.307+0000 7f6f9be00640 1 -- 172.21.15.143:0/1717462561 shutdown_connections 2024-01-16T15:00:52.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.307+0000 7f6f9be00640 1 -- 172.21.15.143:0/1717462561 wait complete. 2024-01-16T15:00:52.307 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-16T15:00:52.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.531+0000 7fdd74400640 1 Processor -- start 2024-01-16T15:00:52.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.531+0000 7fdd74400640 1 -- start start 2024-01-16T15:00:52.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.533+0000 7fdd74400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdd6c06de10 0x7fdd6c15e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:52.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.533+0000 7fdd74400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd6c0c49f0 0x7fdd6c0705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:52.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.533+0000 7fdd74400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd6c15ec90 con 0x7fdd6c072160 2024-01-16T15:00:52.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.533+0000 7fdd74400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd6c0cd530 con 0x7fdd6c06de10 2024-01-16T15:00:52.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.533+0000 7fdd74400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c0c4ed0 con 0x7fdd6c0c49f0 2024-01-16T15:00:52.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.534+0000 7fdd72400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd6c0c49f0 0x7fdd6c0705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:52.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.534+0000 7fdd72400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd6c0c49f0 0x7fdd6c0705c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:39660/0 (socket says 172.21.15.143:39660) 2024-01-16T15:00:52.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.534+0000 7fdd72400640 1 -- 172.21.15.143:0/3005508335 learned_addr learned my addr 172.21.15.143:0/3005508335 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:52.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.534+0000 7fdd71000640 1 --2- 172.21.15.143:0/3005508335 >> v2:172.21.15.204:3300/0 conn(0x7fdd6c06de10 0x7fdd6c15e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:52.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.534+0000 7fdd72400640 1 -- 172.21.15.143:0/3005508335 >> v2:172.21.15.204:3300/0 conn(0x7fdd6c06de10 msgr2=0x7fdd6c15e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:52.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.534+0000 7fdd72400640 1 --2- 172.21.15.143:0/3005508335 >> v2:172.21.15.204:3300/0 conn(0x7fdd6c06de10 0x7fdd6c15e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:52.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.534+0000 7fdd72400640 1 -- 172.21.15.143:0/3005508335 >> v1:172.21.15.204:6789/0 conn(0x7fdd6c072160 legacy=0x7fdd6c06d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:52.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.534+0000 7fdd72400640 1 -- 172.21.15.143:0/3005508335 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6c0c51b0 con 0x7fdd6c0c49f0 2024-01-16T15:00:52.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.536+0000 7fdd72400640 1 --2- 172.21.15.143:0/3005508335 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd6c0c49f0 0x7fdd6c0705c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdd64009870 tx=0x7fdd6402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d56e8b69434eb46a server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:52.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.536+0000 7fdd6be00640 1 -- 172.21.15.143:0/3005508335 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd64030840 con 0x7fdd6c0c49f0 2024-01-16T15:00:52.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.537+0000 7fdd6be00640 1 -- 172.21.15.143:0/3005508335 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd640309a0 con 0x7fdd6c0c49f0 2024-01-16T15:00:52.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.537+0000 7fdd6be00640 1 -- 172.21.15.143:0/3005508335 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd64030d30 con 0x7fdd6c0c49f0 2024-01-16T15:00:52.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.537+0000 7fdd74400640 1 -- 172.21.15.143:0/3005508335 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd6c0c49f0 msgr2=0x7fdd6c0705c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:52.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.537+0000 7fdd74400640 1 --2- 172.21.15.143:0/3005508335 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd6c0c49f0 0x7fdd6c0705c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdd64009870 tx=0x7fdd6402b980 comp rx=0 tx=0).stop 2024-01-16T15:00:52.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.537+0000 7fdd74400640 1 -- 172.21.15.143:0/3005508335 shutdown_connections 2024-01-16T15:00:52.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.537+0000 7fdd74400640 1 --2- 172.21.15.143:0/3005508335 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd6c0c49f0 0x7fdd6c0705c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:52.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.537+0000 7fdd74400640 1 --2- 172.21.15.143:0/3005508335 >> v2:172.21.15.204:3300/0 conn(0x7fdd6c06de10 0x7fdd6c15e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:52.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.537+0000 7fdd74400640 1 -- 172.21.15.143:0/3005508335 >> 172.21.15.143:0/3005508335 conn(0x7fdd6c0b0c50 msgr2=0x7fdd6c0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:52.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.537+0000 7fdd74400640 1 -- 172.21.15.143:0/3005508335 shutdown_connections 2024-01-16T15:00:52.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.537+0000 7fdd74400640 1 -- 172.21.15.143:0/3005508335 wait complete. 2024-01-16T15:00:52.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.539+0000 7fdd74400640 1 Processor -- start 2024-01-16T15:00:52.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.539+0000 7fdd74400640 1 -- start start 2024-01-16T15:00:52.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.540+0000 7fdd74400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd6c06de10 0x7fdd6c1543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:52.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.540+0000 7fdd74400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd6c072160 0x7fdd6c159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:52.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.540+0000 7fdd74400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd6c0c49f0 0x7fdd6c0af150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:52.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.540+0000 7fdd74400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c153310 con 0x7fdd6c072160 2024-01-16T15:00:52.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.540+0000 7fdd74400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c1535f0 con 0x7fdd6c06de10 2024-01-16T15:00:52.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.540+0000 7fdd74400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c1538d0 con 0x7fdd6c0c49f0 2024-01-16T15:00:52.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.540+0000 7fdd71000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd6c072160 0x7fdd6c159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:52.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.540+0000 7fdd71000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd6c072160 0x7fdd6c159740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59944/0 (socket says 172.21.15.143:59944) 2024-01-16T15:00:52.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.540+0000 7fdd71000640 1 -- 172.21.15.143:0/1059682317 learned_addr learned my addr 172.21.15.143:0/1059682317 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:52.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.541+0000 7fdd71a00640 1 --2- 172.21.15.143:0/1059682317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd6c06de10 0x7fdd6c1543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:52.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.541+0000 7fdd72400640 1 --2- 172.21.15.143:0/1059682317 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd6c0c49f0 0x7fdd6c0af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:52.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.541+0000 7fdd71000640 1 -- 172.21.15.143:0/1059682317 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd6c0c49f0 msgr2=0x7fdd6c0af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:52.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.541+0000 7fdd71000640 1 --2- 172.21.15.143:0/1059682317 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd6c0c49f0 0x7fdd6c0af150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:52.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.541+0000 7fdd71000640 1 -- 172.21.15.143:0/1059682317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd6c06de10 msgr2=0x7fdd6c1543b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:52.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.541+0000 7fdd71000640 1 --2- 172.21.15.143:0/1059682317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd6c06de10 0x7fdd6c1543b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:52.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.541+0000 7fdd71000640 1 -- 172.21.15.143:0/1059682317 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd640094d0 con 0x7fdd6c072160 2024-01-16T15:00:52.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.541+0000 7fdd71000640 1 --2- 172.21.15.143:0/1059682317 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd6c072160 0x7fdd6c159740 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdd5400e9b0 tx=0x7fdd5400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:52.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.541+0000 7fdd69600640 1 -- 172.21.15.143:0/1059682317 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd54009670 con 0x7fdd6c072160 2024-01-16T15:00:52.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.541+0000 7fdd69600640 1 -- 172.21.15.143:0/1059682317 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5400c9a0 con 0x7fdd6c072160 2024-01-16T15:00:52.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.541+0000 7fdd69600640 1 -- 172.21.15.143:0/1059682317 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5400cd50 con 0x7fdd6c072160 2024-01-16T15:00:52.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.541+0000 7fdd74400640 1 -- 172.21.15.143:0/1059682317 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd6c153bb0 con 0x7fdd6c072160 2024-01-16T15:00:52.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.542+0000 7fdd74400640 1 -- 172.21.15.143:0/1059682317 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd6c1548f0 con 0x7fdd6c072160 2024-01-16T15:00:52.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.543+0000 7fdd74400640 1 -- 172.21.15.143:0/1059682317 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd6c04e8b0 con 0x7fdd6c072160 2024-01-16T15:00:52.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.543+0000 7fdd69600640 1 -- 172.21.15.143:0/1059682317 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fdd54013470 con 0x7fdd6c072160 2024-01-16T15:00:52.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.544+0000 7fdd69600640 1 --2- 172.21.15.143:0/1059682317 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fdd4406b2f0 0x7fdd4406d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:52.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.544+0000 7fdd69600640 1 -- 172.21.15.143:0/1059682317 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fdd54089e20 con 0x7fdd6c072160 2024-01-16T15:00:52.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.545+0000 7fdd71a00640 1 --2- 172.21.15.143:0/1059682317 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fdd4406b2f0 0x7fdd4406d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:52.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.545+0000 7fdd71a00640 1 --2- 172.21.15.143:0/1059682317 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fdd4406b2f0 0x7fdd4406d7b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c0c47a0 tx=0x7fdd5c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:52.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.549+0000 7fdd69600640 1 -- 172.21.15.143:0/1059682317 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdd54054c00 con 0x7fdd6c072160 2024-01-16T15:00:52.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.747+0000 7fdd74400640 1 -- 172.21.15.143:0/1059682317 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fdd6c15dab0 con 0x7fdd6c072160 2024-01-16T15:00:52.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.748+0000 7fdd69600640 1 -- 172.21.15.143:0/1059682317 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fdd54017080 con 0x7fdd6c072160 2024-01-16T15:00:52.729 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2024-01-16T15:00:52.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.750+0000 7fdd74400640 1 -- 172.21.15.143:0/1059682317 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fdd4406b2f0 msgr2=0x7fdd4406d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:52.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.750+0000 7fdd74400640 1 --2- 172.21.15.143:0/1059682317 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fdd4406b2f0 0x7fdd4406d7b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c0c47a0 tx=0x7fdd5c009040 comp rx=0 tx=0).stop 2024-01-16T15:00:52.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.750+0000 7fdd74400640 1 -- 172.21.15.143:0/1059682317 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd6c072160 msgr2=0x7fdd6c159740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:52.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.750+0000 7fdd74400640 1 --2- 172.21.15.143:0/1059682317 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd6c072160 0x7fdd6c159740 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdd5400e9b0 tx=0x7fdd5400ee70 comp rx=0 tx=0).stop 2024-01-16T15:00:52.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.750+0000 7fdd74400640 1 -- 172.21.15.143:0/1059682317 shutdown_connections 2024-01-16T15:00:52.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.750+0000 7fdd74400640 1 --2- 172.21.15.143:0/1059682317 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fdd4406b2f0 0x7fdd4406d7b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:52.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.750+0000 7fdd74400640 1 --2- 172.21.15.143:0/1059682317 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd6c0c49f0 0x7fdd6c0af150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:52.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.750+0000 7fdd74400640 1 --2- 172.21.15.143:0/1059682317 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd6c072160 0x7fdd6c159740 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:52.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.750+0000 7fdd74400640 1 --2- 172.21.15.143:0/1059682317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd6c06de10 0x7fdd6c1543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:52.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.750+0000 7fdd74400640 1 -- 172.21.15.143:0/1059682317 >> 172.21.15.143:0/1059682317 conn(0x7fdd6c0b0c50 msgr2=0x7fdd6c0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:52.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.751+0000 7fdd74400640 1 -- 172.21.15.143:0/1059682317 shutdown_connections 2024-01-16T15:00:52.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:52.751+0000 7fdd74400640 1 -- 172.21.15.143:0/1059682317 wait complete. 2024-01-16T15:00:52.748 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-01-16T15:00:52.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:52.748 DEBUG:teuthology.orchestra.run.smithi143:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-01-16T15:00:52.836 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2024-01-16T15:00:52.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:52.836 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-01-16T15:00:52.883 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-16T15:00:52.883 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T15:00:52.940 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-16T15:00:52.940 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T15:00:52.994 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-16T15:00:52.994 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T15:00:53.046 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T15:00:53.124 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:00:53.130 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:00:53.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.274+0000 7fc31de00640 1 Processor -- start 2024-01-16T15:00:53.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.274+0000 7fc31de00640 1 -- start start 2024-01-16T15:00:53.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.275+0000 7fc31de00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc31806aa30 0x7fc3180cc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:53.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.275+0000 7fc31de00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc3180b28d0 0x7fc3180cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:53.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.276+0000 7fc31de00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3180b2690 con 0x7fc3180cf420 2024-01-16T15:00:53.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.276+0000 7fc31de00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3180cc7f0 con 0x7fc3180b28d0 2024-01-16T15:00:53.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.276+0000 7fc31de00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc31806ed00 con 0x7fc31806aa30 2024-01-16T15:00:53.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.276+0000 7fc317400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc3180cf420 0x7fc3181616d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50590/0 (socket says 172.21.15.143:50590) 2024-01-16T15:00:53.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.276+0000 7fc317400640 1 -- 172.21.15.143:0/2790252217 learned_addr learned my addr 172.21.15.143:0/2790252217 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:53.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.276+0000 7fc30de00640 1 --2- 172.21.15.143:0/2790252217 >> v2:172.21.15.204:3300/0 conn(0x7fc3180b28d0 0x7fc3180cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:53.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.276+0000 7fc316a00640 1 --2- 172.21.15.143:0/2790252217 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc31806aa30 0x7fc3180cc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:53.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.276+0000 7fc316000640 1 -- 172.21.15.143:0/2790252217 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4137802150 0 0) 0x7fc3180b2690 con 0x7fc3180cf420 2024-01-16T15:00:53.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.276+0000 7fc316000640 1 -- 172.21.15.143:0/2790252217 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2f8003520 con 0x7fc3180cf420 2024-01-16T15:00:53.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.277+0000 7fc30de00640 1 -- 172.21.15.143:0/2790252217 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc31806aa30 msgr2=0x7fc3180cc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:53.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.277+0000 7fc30de00640 1 --2- 172.21.15.143:0/2790252217 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc31806aa30 0x7fc3180cc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:53.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.277+0000 7fc30de00640 1 -- 172.21.15.143:0/2790252217 >> v1:172.21.15.143:6789/0 conn(0x7fc3180cf420 legacy=0x7fc3181616d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:53.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.277+0000 7fc30de00640 1 -- 172.21.15.143:0/2790252217 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc31806efe0 con 0x7fc3180b28d0 2024-01-16T15:00:53.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.278+0000 7fc30de00640 1 --2- 172.21.15.143:0/2790252217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3180b28d0 0x7fc3180cc990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc3000097a0 tx=0x7fc30002b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d333ee3e9ff042b2 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:53.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.278+0000 7fc316000640 1 -- 172.21.15.143:0/2790252217 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc300030800 con 0x7fc3180b28d0 2024-01-16T15:00:53.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.279+0000 7fc316000640 1 -- 172.21.15.143:0/2790252217 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc300030960 con 0x7fc3180b28d0 2024-01-16T15:00:53.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.279+0000 7fc316000640 1 -- 172.21.15.143:0/2790252217 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc300030cf0 con 0x7fc3180b28d0 2024-01-16T15:00:53.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.279+0000 7fc31de00640 1 -- 172.21.15.143:0/2790252217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3180b28d0 msgr2=0x7fc3180cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:53.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.279+0000 7fc31de00640 1 --2- 172.21.15.143:0/2790252217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3180b28d0 0x7fc3180cc990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc3000097a0 tx=0x7fc30002b940 comp rx=0 tx=0).stop 2024-01-16T15:00:53.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.279+0000 7fc31de00640 1 -- 172.21.15.143:0/2790252217 shutdown_connections 2024-01-16T15:00:53.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.279+0000 7fc31de00640 1 --2- 172.21.15.143:0/2790252217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3180b28d0 0x7fc3180cc990 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:53.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.279+0000 7fc31de00640 1 --2- 172.21.15.143:0/2790252217 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc31806aa30 0x7fc3180cc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:53.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.279+0000 7fc31de00640 1 -- 172.21.15.143:0/2790252217 >> 172.21.15.143:0/2790252217 conn(0x7fc3180999d0 msgr2=0x7fc3180b1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:53.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.279+0000 7fc31de00640 1 -- 172.21.15.143:0/2790252217 shutdown_connections 2024-01-16T15:00:53.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.279+0000 7fc31de00640 1 -- 172.21.15.143:0/2790252217 wait complete. 2024-01-16T15:00:53.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.281+0000 7fc31de00640 1 Processor -- start 2024-01-16T15:00:53.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.281+0000 7fc31de00640 1 -- start start 2024-01-16T15:00:53.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.282+0000 7fc31de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc31806aa30 0x7fc3181f2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:53.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.282+0000 7fc31de00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3180b28d0 0x7fc31815da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:53.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.282+0000 7fc31de00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3180cf420 0x7fc31815dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:53.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.282+0000 7fc31de00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3180b4690 con 0x7fc31806aa30 2024-01-16T15:00:53.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.282+0000 7fc31de00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31815e5d0 con 0x7fc3180b28d0 2024-01-16T15:00:53.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.282+0000 7fc31de00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc31815e8b0 con 0x7fc3180cf420 2024-01-16T15:00:53.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.282+0000 7fc316a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc31806aa30 0x7fc3181f2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:53.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.282+0000 7fc316a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc31806aa30 0x7fc3181f2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59948/0 (socket says 172.21.15.143:59948) 2024-01-16T15:00:53.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.282+0000 7fc316a00640 1 -- 172.21.15.143:0/2501549602 learned_addr learned my addr 172.21.15.143:0/2501549602 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:53.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.283+0000 7fc30de00640 1 --2- 172.21.15.143:0/2501549602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3180b28d0 0x7fc31815da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:53.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.283+0000 7fc317400640 1 --2- 172.21.15.143:0/2501549602 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3180cf420 0x7fc31815dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:53.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.283+0000 7fc316a00640 1 -- 172.21.15.143:0/2501549602 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3180cf420 msgr2=0x7fc31815dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:53.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.283+0000 7fc316a00640 1 --2- 172.21.15.143:0/2501549602 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3180cf420 0x7fc31815dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:53.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.283+0000 7fc316a00640 1 -- 172.21.15.143:0/2501549602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3180b28d0 msgr2=0x7fc31815da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:53.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.283+0000 7fc316a00640 1 --2- 172.21.15.143:0/2501549602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3180b28d0 0x7fc31815da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:53.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.283+0000 7fc316a00640 1 -- 172.21.15.143:0/2501549602 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc300009400 con 0x7fc31806aa30 2024-01-16T15:00:53.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.283+0000 7fc316a00640 1 --2- 172.21.15.143:0/2501549602 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc31806aa30 0x7fc3181f2860 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc30800efc0 tx=0x7fc30800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:53.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.284+0000 7fc30fe00640 1 -- 172.21.15.143:0/2501549602 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc308014480 con 0x7fc31806aa30 2024-01-16T15:00:53.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.284+0000 7fc30fe00640 1 -- 172.21.15.143:0/2501549602 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc30800f040 con 0x7fc31806aa30 2024-01-16T15:00:53.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.284+0000 7fc30fe00640 1 -- 172.21.15.143:0/2501549602 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc308014800 con 0x7fc31806aa30 2024-01-16T15:00:53.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.284+0000 7fc31de00640 1 -- 172.21.15.143:0/2501549602 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc31815eb90 con 0x7fc31806aa30 2024-01-16T15:00:53.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.284+0000 7fc31de00640 1 -- 172.21.15.143:0/2501549602 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3181f2da0 con 0x7fc31806aa30 2024-01-16T15:00:53.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.285+0000 7fc31de00640 1 -- 172.21.15.143:0/2501549602 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3180c71d0 con 0x7fc31806aa30 2024-01-16T15:00:53.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.286+0000 7fc30fe00640 1 -- 172.21.15.143:0/2501549602 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fc308014960 con 0x7fc31806aa30 2024-01-16T15:00:53.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.287+0000 7fc30fe00640 1 --2- 172.21.15.143:0/2501549602 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fc2f006b2c0 0x7fc2f006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:53.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.287+0000 7fc30fe00640 1 -- 172.21.15.143:0/2501549602 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fc30808a860 con 0x7fc31806aa30 2024-01-16T15:00:53.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.287+0000 7fc30de00640 1 --2- 172.21.15.143:0/2501549602 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fc2f006b2c0 0x7fc2f006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:53.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.288+0000 7fc30de00640 1 --2- 172.21.15.143:0/2501549602 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fc2f006b2c0 0x7fc2f006d780 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc300036000 tx=0x7fc300008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:53.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.291+0000 7fc30fe00640 1 -- 172.21.15.143:0/2501549602 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc308055640 con 0x7fc31806aa30 2024-01-16T15:00:53.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.485+0000 7fc31de00640 1 -- 172.21.15.143:0/2501549602 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc31806e710 con 0x7fc31806aa30 2024-01-16T15:00:53.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.486+0000 7fc30fe00640 1 -- 172.21.15.143:0/2501549602 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16814 (secure 0 0 0) 0x7fc308058c60 con 0x7fc31806aa30 2024-01-16T15:00:53.467 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:00:53.467 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":22,"fsid":"c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0","created":"2024-01-16T14:59:38.532601+0000","modified":"2024-01-16T15:00:45.626415+0000","last_up_change":"2024-01-16T14:59:50.983699+0000","last_in_change":"2024-01-16T14:59:42.056297+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:59:51.727720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T15:00:01.010472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T15:00:02.041228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f4c8891d-ab29-472a-907d-8446f20f7012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6801","nonce":1569293626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6805","nonce":1569293626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6814","nonce":1569293626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6809","nonce":1569293626}]},"public_addr":"172.21.15.143:6801/1569293626","cluster_addr":"172.21.15.143:6805/1569293626","heartbeat_back_addr":"172.21.15.143:6814/1569293626","heartbeat_front_addr":"172.21.15.143:6809/1569293626","state":["exists","up"]},{"osd":1,"uuid":"c4a2788d-8b89-4992-82b2-7f4b9debd509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6804","nonce":1276841168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6808","nonce":1276841168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6813","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6815","nonce":1276841168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6811","nonce":1276841168}]},"public_addr":"172.21.15.143:6804/1276841168","cluster_addr":"172.21.15.143:6808/1276841168","heartbeat_back_addr":"172.21.15.143:6815/1276841168","heartbeat_front_addr":"172.21.15.143:6811/1276841168","state":["exists","up"]},{"osd":2,"uuid":"64645c1d-31b2-4794-b0f2-7ccf8df30393","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6817","nonce":1217670170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6819","nonce":1217670170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6823","nonce":1217670170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6821","nonce":1217670170}]},"public_addr":"172.21.15.143:6817/1217670170","cluster_addr":"172.21.15.143:6819/1217670170","heartbeat_back_addr":"172.21.15.143:6823/1217670170","heartbeat_front_addr":"172.21.15.143:6821/1217670170","state":["exists","up"]},{"osd":3,"uuid":"73674129-a84e-4d31-8247-7d0179ae6df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6825","nonce":3414188012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6827","nonce":3414188012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6831","nonce":3414188012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6829","nonce":3414188012}]},"public_addr":"172.21.15.143:6825/3414188012","cluster_addr":"172.21.15.143:6827/3414188012","heartbeat_back_addr":"172.21.15.143:6831/3414188012","heartbeat_front_addr":"172.21.15.143:6829/3414188012","state":["exists","up"]},{"osd":4,"uuid":"cea28c9b-2f1c-4087-b415-9960789131d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6811","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6812","nonce":1070255716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6816","nonce":1070255716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6823","nonce":1070255716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6820","nonce":1070255716}]},"public_addr":"172.21.15.204:6812/1070255716","cluster_addr":"172.21.15.204:6816/1070255716","heartbeat_back_addr":"172.21.15.204:6823/1070255716","heartbeat_front_addr":"172.21.15.204:6820/1070255716","state":["exists","up"]},{"osd":5,"uuid":"33b63560-5574-4e78-983f-b398d67a2cb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6809","nonce":3605417837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6813","nonce":3605417837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6819","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6821","nonce":3605417837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6817","nonce":3605417837}]},"public_addr":"172.21.15.204:6809/3605417837","cluster_addr":"172.21.15.204:6813/3605417837","heartbeat_back_addr":"172.21.15.204:6821/3605417837","heartbeat_front_addr":"172.21.15.204:6817/3605417837","state":["exists","up"]},{"osd":6,"uuid":"df397427-8940-4eee-9c79-eece669d735d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6801","nonce":1085000534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6803","nonce":1085000534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6808","nonce":1085000534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6805","nonce":1085000534}]},"public_addr":"172.21.15.204:6801/1085000534","cluster_addr":"172.21.15.204:6803/1085000534","heartbeat_back_addr":"172.21.15.204:6808/1085000534","heartbeat_front_addr":"172.21.15.204:6805/1085000534","state":["exists","up"]},{"osd":7,"uuid":"a2e5a15c-8312-405a-ac02-f7e1fc9df2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6825","nonce":1337220609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6827","nonce":1337220609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6831","nonce":1337220609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6829","nonce":1337220609}]},"public_addr":"172.21.15.204:6825/1337220609","cluster_addr":"172.21.15.204:6827/1337220609","heartbeat_back_addr":"172.21.15.204:6831/1337220609","heartbeat_front_addr":"172.21.15.204:6829/1337220609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:49.984507+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.395306+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.632998+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.373627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.342920+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.438366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.331926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.406657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T15:00:53.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.489+0000 7fc31de00640 1 -- 172.21.15.143:0/2501549602 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fc2f006b2c0 msgr2=0x7fc2f006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:53.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.489+0000 7fc31de00640 1 --2- 172.21.15.143:0/2501549602 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fc2f006b2c0 0x7fc2f006d780 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc300036000 tx=0x7fc300008030 comp rx=0 tx=0).stop 2024-01-16T15:00:53.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.489+0000 7fc31de00640 1 -- 172.21.15.143:0/2501549602 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc31806aa30 msgr2=0x7fc3181f2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:53.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.489+0000 7fc31de00640 1 --2- 172.21.15.143:0/2501549602 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc31806aa30 0x7fc3181f2860 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc30800efc0 tx=0x7fc30800c490 comp rx=0 tx=0).stop 2024-01-16T15:00:53.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.490+0000 7fc31de00640 1 -- 172.21.15.143:0/2501549602 shutdown_connections 2024-01-16T15:00:53.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.490+0000 7fc31de00640 1 --2- 172.21.15.143:0/2501549602 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fc2f006b2c0 0x7fc2f006d780 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:53.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.490+0000 7fc31de00640 1 --2- 172.21.15.143:0/2501549602 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3180cf420 0x7fc31815dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:53.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.490+0000 7fc31de00640 1 --2- 172.21.15.143:0/2501549602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3180b28d0 0x7fc31815da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:53.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.490+0000 7fc31de00640 1 --2- 172.21.15.143:0/2501549602 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc31806aa30 0x7fc3181f2860 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:53.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.490+0000 7fc31de00640 1 -- 172.21.15.143:0/2501549602 >> 172.21.15.143:0/2501549602 conn(0x7fc3180999d0 msgr2=0x7fc3181607e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:53.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.490+0000 7fc31de00640 1 -- 172.21.15.143:0/2501549602 shutdown_connections 2024-01-16T15:00:53.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.490+0000 7fc31de00640 1 -- 172.21.15.143:0/2501549602 wait complete. 2024-01-16T15:00:53.498 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-16T14:59:51.727720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-16T15:00:01.010472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-16T15:00:02.041228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-16T15:00:53.499 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-16T15:00:53.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.738+0000 7fa986000640 1 Processor -- start 2024-01-16T15:00:53.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.738+0000 7fa986000640 1 -- start start 2024-01-16T15:00:53.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.740+0000 7fa986000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa98015d3f0 0x7fa9800d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:53.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.740+0000 7fa986000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa98015d7d0 0x7fa9800d1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:53.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.740+0000 7fa986000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9800d15e0 con 0x7fa9800d0e40 2024-01-16T15:00:53.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.740+0000 7fa986000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9800d0280 con 0x7fa98015d3f0 2024-01-16T15:00:53.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.740+0000 7fa986000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9800d1300 con 0x7fa98015d7d0 2024-01-16T15:00:53.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.740+0000 7fa984c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa98015d3f0 0x7fa9800d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:53.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.740+0000 7fa985600640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fa9800d0e40 0x7fa9800afea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50600/0 (socket says 172.21.15.143:50600) 2024-01-16T15:00:53.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.740+0000 7fa985600640 1 -- 172.21.15.143:0/3573994188 learned_addr learned my addr 172.21.15.143:0/3573994188 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:53.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.741+0000 7fa97fe00640 1 --2- 172.21.15.143:0/3573994188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa98015d7d0 0x7fa9800d1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:53.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.741+0000 7fa984c00640 1 -- 172.21.15.143:0/3573994188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa98015d7d0 msgr2=0x7fa9800d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:53.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.741+0000 7fa984c00640 1 --2- 172.21.15.143:0/3573994188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa98015d7d0 0x7fa9800d1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:53.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.741+0000 7fa984c00640 1 -- 172.21.15.143:0/3573994188 >> v1:172.21.15.143:6789/0 conn(0x7fa9800d0e40 legacy=0x7fa9800afea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:53.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.741+0000 7fa984c00640 1 -- 172.21.15.143:0/3573994188 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9800cfc30 con 0x7fa98015d3f0 2024-01-16T15:00:53.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.741+0000 7fa97f400640 1 -- 172.21.15.143:0/3573994188 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2809085934 0 0) 0x7fa9800d15e0 con 0x7fa9800d0e40 2024-01-16T15:00:53.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.742+0000 7fa984c00640 1 --2- 172.21.15.143:0/3573994188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa98015d3f0 0x7fa9800d1830 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa9700097a0 tx=0x7fa97002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abae79a6e8b25c91 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:53.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.742+0000 7fa97f400640 1 -- 172.21.15.143:0/3573994188 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa970031830 con 0x7fa98015d3f0 2024-01-16T15:00:53.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.742+0000 7fa97f400640 1 -- 172.21.15.143:0/3573994188 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa970031990 con 0x7fa98015d3f0 2024-01-16T15:00:53.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.742+0000 7fa97f400640 1 -- 172.21.15.143:0/3573994188 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa970031d20 con 0x7fa98015d3f0 2024-01-16T15:00:53.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.743+0000 7fa986000640 1 -- 172.21.15.143:0/3573994188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa98015d3f0 msgr2=0x7fa9800d1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:53.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.743+0000 7fa986000640 1 --2- 172.21.15.143:0/3573994188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa98015d3f0 0x7fa9800d1830 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa9700097a0 tx=0x7fa97002b8d0 comp rx=0 tx=0).stop 2024-01-16T15:00:53.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.743+0000 7fa986000640 1 -- 172.21.15.143:0/3573994188 shutdown_connections 2024-01-16T15:00:53.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.743+0000 7fa986000640 1 --2- 172.21.15.143:0/3573994188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa98015d7d0 0x7fa9800d1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:53.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.743+0000 7fa986000640 1 --2- 172.21.15.143:0/3573994188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa98015d3f0 0x7fa9800d1830 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:53.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.743+0000 7fa986000640 1 -- 172.21.15.143:0/3573994188 >> 172.21.15.143:0/3573994188 conn(0x7fa98009c5b0 msgr2=0x7fa9800d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:53.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.743+0000 7fa986000640 1 -- 172.21.15.143:0/3573994188 shutdown_connections 2024-01-16T15:00:53.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.743+0000 7fa986000640 1 -- 172.21.15.143:0/3573994188 wait complete. 2024-01-16T15:00:53.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.745+0000 7fa986000640 1 Processor -- start 2024-01-16T15:00:53.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.745+0000 7fa986000640 1 -- start start 2024-01-16T15:00:53.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.746+0000 7fa986000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9800d0e40 0x7fa9800b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:53.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.746+0000 7fa986000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa98015d3f0 0x7fa9800b82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:53.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.746+0000 7fa986000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa98015d7d0 0x7fa9801f2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:53.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.746+0000 7fa986000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9801f3b70 con 0x7fa98015d3f0 2024-01-16T15:00:53.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.746+0000 7fa986000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9800c7240 con 0x7fa98015d7d0 2024-01-16T15:00:53.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.746+0000 7fa986000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9801f35b0 con 0x7fa9800d0e40 2024-01-16T15:00:53.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.746+0000 7fa97fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa98015d3f0 0x7fa9800b82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:53.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.746+0000 7fa97fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa98015d3f0 0x7fa9800b82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59968/0 (socket says 172.21.15.143:59968) 2024-01-16T15:00:53.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.746+0000 7fa97fe00640 1 -- 172.21.15.143:0/1937188269 learned_addr learned my addr 172.21.15.143:0/1937188269 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:53.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.746+0000 7fa984c00640 1 --2- 172.21.15.143:0/1937188269 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9800d0e40 0x7fa9800b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:53.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.746+0000 7fa97fe00640 1 -- 172.21.15.143:0/1937188269 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9800d0e40 msgr2=0x7fa9800b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:53.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.746+0000 7fa97fe00640 1 --2- 172.21.15.143:0/1937188269 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9800d0e40 0x7fa9800b3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:53.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.746+0000 7fa97fe00640 1 -- 172.21.15.143:0/1937188269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa98015d7d0 msgr2=0x7fa9801f2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:53.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.746+0000 7fa97fe00640 1 --2- 172.21.15.143:0/1937188269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa98015d7d0 0x7fa9801f2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:53.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.746+0000 7fa97fe00640 1 -- 172.21.15.143:0/1937188269 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa970009400 con 0x7fa98015d3f0 2024-01-16T15:00:53.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.747+0000 7fa97fe00640 1 --2- 172.21.15.143:0/1937188269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa98015d3f0 0x7fa9800b82d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa96800d5c0 tx=0x7fa96800da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:53.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.747+0000 7fa97cc00640 1 -- 172.21.15.143:0/1937188269 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9680125f0 con 0x7fa98015d3f0 2024-01-16T15:00:53.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.747+0000 7fa97cc00640 1 -- 172.21.15.143:0/1937188269 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa968012750 con 0x7fa98015d3f0 2024-01-16T15:00:53.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.747+0000 7fa97cc00640 1 -- 172.21.15.143:0/1937188269 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa968012b00 con 0x7fa98015d3f0 2024-01-16T15:00:53.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.747+0000 7fa986000640 1 -- 172.21.15.143:0/1937188269 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9801f3890 con 0x7fa98015d3f0 2024-01-16T15:00:53.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.747+0000 7fa986000640 1 -- 172.21.15.143:0/1937188269 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9800b4090 con 0x7fa98015d3f0 2024-01-16T15:00:53.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.748+0000 7fa986000640 1 -- 172.21.15.143:0/1937188269 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9800d06f0 con 0x7fa98015d3f0 2024-01-16T15:00:53.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.749+0000 7fa97cc00640 1 -- 172.21.15.143:0/1937188269 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fa968012c60 con 0x7fa98015d3f0 2024-01-16T15:00:53.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.750+0000 7fa97cc00640 1 --2- 172.21.15.143:0/1937188269 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fa95806b2f0 0x7fa95806d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:53.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.750+0000 7fa97cc00640 1 -- 172.21.15.143:0/1937188269 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fa96808a090 con 0x7fa98015d3f0 2024-01-16T15:00:53.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.750+0000 7fa984c00640 1 --2- 172.21.15.143:0/1937188269 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fa95806b2f0 0x7fa95806d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:53.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.751+0000 7fa984c00640 1 --2- 172.21.15.143:0/1937188269 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fa95806b2f0 0x7fa95806d7b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa970036000 tx=0x7fa970008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:53.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.755+0000 7fa97cc00640 1 -- 172.21.15.143:0/1937188269 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa9800d06f0 con 0x7fa98015d3f0 2024-01-16T15:00:53.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.942+0000 7fa986000640 1 -- 172.21.15.143:0/1937188269 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa9800d06f0 con 0x7fa98015d3f0 2024-01-16T15:00:53.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.942+0000 7fa97cc00640 1 -- 172.21.15.143:0/1937188269 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa9800d06f0 con 0x7fa98015d3f0 2024-01-16T15:00:53.923 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-16T15:00:53.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.945+0000 7fa986000640 1 -- 172.21.15.143:0/1937188269 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fa95806b2f0 msgr2=0x7fa95806d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:53.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.945+0000 7fa986000640 1 --2- 172.21.15.143:0/1937188269 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fa95806b2f0 0x7fa95806d7b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa970036000 tx=0x7fa970008030 comp rx=0 tx=0).stop 2024-01-16T15:00:53.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.945+0000 7fa986000640 1 -- 172.21.15.143:0/1937188269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa98015d3f0 msgr2=0x7fa9800b82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:53.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.945+0000 7fa986000640 1 --2- 172.21.15.143:0/1937188269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa98015d3f0 0x7fa9800b82d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa96800d5c0 tx=0x7fa96800da80 comp rx=0 tx=0).stop 2024-01-16T15:00:53.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.945+0000 7fa986000640 1 -- 172.21.15.143:0/1937188269 shutdown_connections 2024-01-16T15:00:53.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.945+0000 7fa986000640 1 --2- 172.21.15.143:0/1937188269 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fa95806b2f0 0x7fa95806d7b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:53.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.945+0000 7fa986000640 1 --2- 172.21.15.143:0/1937188269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa98015d7d0 0x7fa9801f2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:53.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.945+0000 7fa986000640 1 --2- 172.21.15.143:0/1937188269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa98015d3f0 0x7fa9800b82d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:53.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.945+0000 7fa986000640 1 --2- 172.21.15.143:0/1937188269 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9800d0e40 0x7fa9800b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:53.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.945+0000 7fa986000640 1 -- 172.21.15.143:0/1937188269 >> 172.21.15.143:0/1937188269 conn(0x7fa98009c5b0 msgr2=0x7fa98006c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:53.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.945+0000 7fa986000640 1 -- 172.21.15.143:0/1937188269 shutdown_connections 2024-01-16T15:00:53.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:53.945+0000 7fa986000640 1 -- 172.21.15.143:0/1937188269 wait complete. 2024-01-16T15:00:53.943 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-16T15:00:54.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.159+0000 7f0ccd400640 1 Processor -- start 2024-01-16T15:00:54.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.159+0000 7f0ccd400640 1 -- start start 2024-01-16T15:00:54.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.161+0000 7f0ccd400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0cc8072160 0x7f0cc815e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:54.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.161+0000 7f0ccd400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0cc806de10 0x7f0cc80705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:54.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.161+0000 7f0ccd400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cc815ec90 con 0x7f0cc80c49f0 2024-01-16T15:00:54.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.161+0000 7f0ccd400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cc80cd530 con 0x7f0cc806de10 2024-01-16T15:00:54.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.161+0000 7f0ccd400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cc80c4ed0 con 0x7f0cc8072160 2024-01-16T15:00:54.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.161+0000 7f0cc5e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f0cc80c49f0 0x7f0cc806d700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50614/0 (socket says 172.21.15.143:50614) 2024-01-16T15:00:54.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.161+0000 7f0cc5e00640 1 -- 172.21.15.143:0/295132964 learned_addr learned my addr 172.21.15.143:0/295132964 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:54.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.161+0000 7f0cc7200640 1 --2- 172.21.15.143:0/295132964 >> v2:172.21.15.143:3300/0 conn(0x7f0cc806de10 0x7f0cc80705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:54.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.162+0000 7f0cc6800640 1 --2- 172.21.15.143:0/295132964 >> v2:172.21.15.204:3300/0 conn(0x7f0cc8072160 0x7f0cc815e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:54.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.162+0000 7f0cc7200640 1 -- 172.21.15.143:0/295132964 >> v2:172.21.15.204:3300/0 conn(0x7f0cc8072160 msgr2=0x7f0cc815e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:54.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.162+0000 7f0cc7200640 1 --2- 172.21.15.143:0/295132964 >> v2:172.21.15.204:3300/0 conn(0x7f0cc8072160 0x7f0cc815e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:54.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.162+0000 7f0cc7200640 1 -- 172.21.15.143:0/295132964 >> v1:172.21.15.143:6789/0 conn(0x7f0cc80c49f0 legacy=0x7f0cc806d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:54.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.162+0000 7f0cc7200640 1 -- 172.21.15.143:0/295132964 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc80c51b0 con 0x7f0cc806de10 2024-01-16T15:00:54.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.163+0000 7f0cc7200640 1 --2- 172.21.15.143:0/295132964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cc806de10 0x7f0cc80705c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0cc80ccae0 tx=0x7f0cc002bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753c2d87aeb381d0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:54.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.163+0000 7f0cc5400640 1 -- 172.21.15.143:0/295132964 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc002cc70 con 0x7f0cc806de10 2024-01-16T15:00:54.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.163+0000 7f0cc5400640 1 -- 172.21.15.143:0/295132964 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cc002cdd0 con 0x7f0cc806de10 2024-01-16T15:00:54.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.163+0000 7f0cc5400640 1 -- 172.21.15.143:0/295132964 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0034660 con 0x7f0cc806de10 2024-01-16T15:00:54.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.164+0000 7f0ccd400640 1 -- 172.21.15.143:0/295132964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cc806de10 msgr2=0x7f0cc80705c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:54.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.164+0000 7f0ccd400640 1 --2- 172.21.15.143:0/295132964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cc806de10 0x7f0cc80705c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0cc80ccae0 tx=0x7f0cc002bec0 comp rx=0 tx=0).stop 2024-01-16T15:00:54.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.164+0000 7f0ccd400640 1 -- 172.21.15.143:0/295132964 shutdown_connections 2024-01-16T15:00:54.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.164+0000 7f0ccd400640 1 --2- 172.21.15.143:0/295132964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cc806de10 0x7f0cc80705c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:54.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.164+0000 7f0ccd400640 1 --2- 172.21.15.143:0/295132964 >> v2:172.21.15.204:3300/0 conn(0x7f0cc8072160 0x7f0cc815e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:54.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.164+0000 7f0ccd400640 1 -- 172.21.15.143:0/295132964 >> 172.21.15.143:0/295132964 conn(0x7f0cc80b0c50 msgr2=0x7f0cc80b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:54.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.164+0000 7f0ccd400640 1 -- 172.21.15.143:0/295132964 shutdown_connections 2024-01-16T15:00:54.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.164+0000 7f0ccd400640 1 -- 172.21.15.143:0/295132964 wait complete. 2024-01-16T15:00:54.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.166+0000 7f0ccd400640 1 Processor -- start 2024-01-16T15:00:54.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.166+0000 7f0ccd400640 1 -- start start 2024-01-16T15:00:54.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.167+0000 7f0ccd400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0cc806de10 0x7f0cc809c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:54.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.167+0000 7f0ccd400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cc8072160 0x7f0cc80abf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:54.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.167+0000 7f0ccd400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cc80c49f0 0x7f0cc80ac470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:54.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.167+0000 7f0ccd400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc80c6170 con 0x7f0cc80c49f0 2024-01-16T15:00:54.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.167+0000 7f0ccd400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc80c63e0 con 0x7f0cc8072160 2024-01-16T15:00:54.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.167+0000 7f0ccd400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc80c66c0 con 0x7f0cc806de10 2024-01-16T15:00:54.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.167+0000 7f0cc7200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cc80c49f0 0x7f0cc80ac470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:54.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.167+0000 7f0cc7200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cc80c49f0 0x7f0cc80ac470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59996/0 (socket says 172.21.15.143:59996) 2024-01-16T15:00:54.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.168+0000 7f0cc7200640 1 -- 172.21.15.143:0/3956490588 learned_addr learned my addr 172.21.15.143:0/3956490588 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:54.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.168+0000 7f0cc6800640 1 --2- 172.21.15.143:0/3956490588 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0cc806de10 0x7f0cc809c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:54.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.168+0000 7f0cc7200640 1 -- 172.21.15.143:0/3956490588 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0cc806de10 msgr2=0x7f0cc809c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:54.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.168+0000 7f0cc7200640 1 --2- 172.21.15.143:0/3956490588 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0cc806de10 0x7f0cc809c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:54.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.168+0000 7f0cc5e00640 1 --2- 172.21.15.143:0/3956490588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cc8072160 0x7f0cc80abf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:54.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.168+0000 7f0cc7200640 1 -- 172.21.15.143:0/3956490588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cc8072160 msgr2=0x7f0cc80abf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:54.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.168+0000 7f0cc7200640 1 --2- 172.21.15.143:0/3956490588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cc8072160 0x7f0cc80abf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:54.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.168+0000 7f0cc7200640 1 -- 172.21.15.143:0/3956490588 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc0009a70 con 0x7f0cc80c49f0 2024-01-16T15:00:54.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.168+0000 7f0cc5e00640 1 --2- 172.21.15.143:0/3956490588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cc8072160 0x7f0cc80abf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:54.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.168+0000 7f0cc7200640 1 --2- 172.21.15.143:0/3956490588 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cc80c49f0 0x7f0cc80ac470 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0009e80 tx=0x7f0cc0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:54.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.168+0000 7f0cbea00640 1 -- 172.21.15.143:0/3956490588 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0040650 con 0x7f0cc80c49f0 2024-01-16T15:00:54.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.169+0000 7f0cbea00640 1 -- 172.21.15.143:0/3956490588 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cc00407d0 con 0x7f0cc80c49f0 2024-01-16T15:00:54.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.169+0000 7f0cbea00640 1 -- 172.21.15.143:0/3956490588 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0040b80 con 0x7f0cc80c49f0 2024-01-16T15:00:54.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.169+0000 7f0ccd400640 1 -- 172.21.15.143:0/3956490588 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cc80c69a0 con 0x7f0cc80c49f0 2024-01-16T15:00:54.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.169+0000 7f0ccd400640 1 -- 172.21.15.143:0/3956490588 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cc80b2a40 con 0x7f0cc80c49f0 2024-01-16T15:00:54.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.170+0000 7f0ccd400640 1 -- 172.21.15.143:0/3956490588 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cc804e8b0 con 0x7f0cc80c49f0 2024-01-16T15:00:54.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.171+0000 7f0cbea00640 1 -- 172.21.15.143:0/3956490588 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f0cc0040ce0 con 0x7f0cc80c49f0 2024-01-16T15:00:54.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.171+0000 7f0cbea00640 1 --2- 172.21.15.143:0/3956490588 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0ca006b2c0 0x7f0ca006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:54.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.172+0000 7f0cbea00640 1 -- 172.21.15.143:0/3956490588 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f0cc00acae0 con 0x7f0cc80c49f0 2024-01-16T15:00:54.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.172+0000 7f0cc6800640 1 --2- 172.21.15.143:0/3956490588 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0ca006b2c0 0x7f0ca006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:54.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.172+0000 7f0cc6800640 1 --2- 172.21.15.143:0/3956490588 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0ca006b2c0 0x7f0ca006d780 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8006fd0 tx=0x7f0cb8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:54.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.180+0000 7f0cbea00640 1 -- 172.21.15.143:0/3956490588 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0cc0077830 con 0x7f0cc80c49f0 2024-01-16T15:00:54.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.369+0000 7f0ccd400640 1 -- 172.21.15.143:0/3956490588 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0cc815dab0 con 0x7f0cc80c49f0 2024-01-16T15:00:54.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.370+0000 7f0cbea00640 1 -- 172.21.15.143:0/3956490588 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0cc007ae50 con 0x7f0cc80c49f0 2024-01-16T15:00:54.350 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2024-01-16T15:00:54.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.372+0000 7f0ccd400640 1 -- 172.21.15.143:0/3956490588 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0ca006b2c0 msgr2=0x7f0ca006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:54.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.372+0000 7f0ccd400640 1 --2- 172.21.15.143:0/3956490588 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0ca006b2c0 0x7f0ca006d780 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8006fd0 tx=0x7f0cb8008040 comp rx=0 tx=0).stop 2024-01-16T15:00:54.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.372+0000 7f0ccd400640 1 -- 172.21.15.143:0/3956490588 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cc80c49f0 msgr2=0x7f0cc80ac470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:54.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.372+0000 7f0ccd400640 1 --2- 172.21.15.143:0/3956490588 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cc80c49f0 0x7f0cc80ac470 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0009e80 tx=0x7f0cc0004a40 comp rx=0 tx=0).stop 2024-01-16T15:00:54.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.372+0000 7f0ccd400640 1 -- 172.21.15.143:0/3956490588 shutdown_connections 2024-01-16T15:00:54.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.372+0000 7f0ccd400640 1 --2- 172.21.15.143:0/3956490588 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0ca006b2c0 0x7f0ca006d780 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:54.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.372+0000 7f0ccd400640 1 --2- 172.21.15.143:0/3956490588 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cc80c49f0 0x7f0cc80ac470 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:54.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.372+0000 7f0ccd400640 1 --2- 172.21.15.143:0/3956490588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cc8072160 0x7f0cc80abf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:54.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.372+0000 7f0ccd400640 1 --2- 172.21.15.143:0/3956490588 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0cc806de10 0x7f0cc809c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:54.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.372+0000 7f0ccd400640 1 -- 172.21.15.143:0/3956490588 >> 172.21.15.143:0/3956490588 conn(0x7f0cc80b0c50 msgr2=0x7f0cc80ae380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:54.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.372+0000 7f0ccd400640 1 -- 172.21.15.143:0/3956490588 shutdown_connections 2024-01-16T15:00:54.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.372+0000 7f0ccd400640 1 -- 172.21.15.143:0/3956490588 wait complete. 2024-01-16T15:00:54.369 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-16T15:00:54.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.571+0000 7f162ac00640 1 Processor -- start 2024-01-16T15:00:54.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.571+0000 7f162ac00640 1 -- start start 2024-01-16T15:00:54.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.573+0000 7f162ac00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1624140340 0x7f16240ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:54.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.573+0000 7f162ac00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1624161080 0x7f16240c60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:54.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.573+0000 7f162ac00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1624161600 con 0x7f1624140720 2024-01-16T15:00:54.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.573+0000 7f162ac00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16240cbf70 con 0x7f1624140340 2024-01-16T15:00:54.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.573+0000 7f162ac00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16241618e0 con 0x7f1624161080 2024-01-16T15:00:54.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.573+0000 7f161be00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f1624140720 0x7f16240cb860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50622/0 (socket says 172.21.15.143:50622) 2024-01-16T15:00:54.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.573+0000 7f161be00640 1 -- 172.21.15.143:0/4249972021 learned_addr learned my addr 172.21.15.143:0/4249972021 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:54.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.574+0000 7f1623e00640 1 --2- 172.21.15.143:0/4249972021 >> v2:172.21.15.204:3300/0 conn(0x7f1624140340 0x7f16240ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:54.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.574+0000 7f1628c00640 1 --2- 172.21.15.143:0/4249972021 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1624161080 0x7f16240c60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:54.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.574+0000 7f1623400640 1 -- 172.21.15.143:0/4249972021 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2970501097 0 0) 0x7f1624161600 con 0x7f1624140720 2024-01-16T15:00:54.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.574+0000 7f1623400640 1 -- 172.21.15.143:0/4249972021 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1604003520 con 0x7f1624140720 2024-01-16T15:00:54.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.574+0000 7f1623400640 1 -- 172.21.15.143:0/4249972021 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 51903287 0 0) 0x7f1604003520 con 0x7f1624140720 2024-01-16T15:00:54.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.574+0000 7f1623400640 1 -- 172.21.15.143:0/4249972021 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1624161080 msgr2=0x7f16240c60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:54.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.574+0000 7f1623400640 1 --2- 172.21.15.143:0/4249972021 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1624161080 0x7f16240c60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:54.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.574+0000 7f1623400640 1 -- 172.21.15.143:0/4249972021 >> v2:172.21.15.204:3300/0 conn(0x7f1624140340 msgr2=0x7f16240ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:54.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.574+0000 7f1623400640 1 --2- 172.21.15.143:0/4249972021 >> v2:172.21.15.204:3300/0 conn(0x7f1624140340 0x7f16240ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:54.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.574+0000 7f1623400640 1 -- 172.21.15.143:0/4249972021 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1624161bc0 con 0x7f1624140720 2024-01-16T15:00:54.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.574+0000 7f1623400640 1 -- 172.21.15.143:0/4249972021 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7f160c001670 con 0x7f1624140720 2024-01-16T15:00:54.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.574+0000 7f1623400640 1 -- 172.21.15.143:0/4249972021 >> v1:172.21.15.143:6789/0 conn(0x7f1624140720 legacy=0x7f16240cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:54.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.574+0000 7f1623400640 1 --2- 172.21.15.143:0/4249972021 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1624161080 0x7f1604003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:54.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.574+0000 7f1623400640 1 --2- 172.21.15.143:0/4249972021 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1604003ef0 0x7f16040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:54.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.575+0000 7f1623400640 1 --2- 172.21.15.143:0/4249972021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1604004d00 0x7f1604005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:54.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.575+0000 7f1623400640 1 -- 172.21.15.143:0/4249972021 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16240cbf70 con 0x7f1604003ef0 2024-01-16T15:00:54.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.575+0000 7f1623400640 1 -- 172.21.15.143:0/4249972021 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16241618e0 con 0x7f1604004d00 2024-01-16T15:00:54.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.575+0000 7f1623400640 1 -- 172.21.15.143:0/4249972021 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f160400b9d0 con 0x7f1624161080 2024-01-16T15:00:54.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.575+0000 7f161be00640 1 --2- 172.21.15.143:0/4249972021 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1604003ef0 0x7f16040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:54.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.575+0000 7f1623e00640 1 --2- 172.21.15.143:0/4249972021 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1624161080 0x7f1604003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:54.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.575+0000 7f1628c00640 1 --2- 172.21.15.143:0/4249972021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1604004d00 0x7f1604005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:54.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.575+0000 7f161be00640 1 -- 172.21.15.143:0/4249972021 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1624161080 msgr2=0x7f1604003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:54.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.575+0000 7f161be00640 1 --2- 172.21.15.143:0/4249972021 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1624161080 0x7f1604003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:54.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.575+0000 7f161be00640 1 -- 172.21.15.143:0/4249972021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1604004d00 msgr2=0x7f1604005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:54.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.575+0000 7f161be00640 1 --2- 172.21.15.143:0/4249972021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1604004d00 0x7f1604005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:54.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.575+0000 7f161be00640 1 -- 172.21.15.143:0/4249972021 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1604003520 con 0x7f1604003ef0 2024-01-16T15:00:54.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.575+0000 7f1623e00640 1 --2- 172.21.15.143:0/4249972021 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1624161080 0x7f1604003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:54.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.577+0000 7f161be00640 1 --2- 172.21.15.143:0/4249972021 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1604003ef0 0x7f16040047c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f160c00c270 tx=0x7f16040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ac74d311a210bf server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:54.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.577+0000 7f1623400640 1 -- 172.21.15.143:0/4249972021 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160c02cae0 con 0x7f1604003ef0 2024-01-16T15:00:54.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.577+0000 7f1623400640 1 -- 172.21.15.143:0/4249972021 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f160c02cc40 con 0x7f1604003ef0 2024-01-16T15:00:54.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.577+0000 7f1623400640 1 -- 172.21.15.143:0/4249972021 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160c005af0 con 0x7f1604003ef0 2024-01-16T15:00:54.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.578+0000 7f162ac00640 1 -- 172.21.15.143:0/4249972021 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1604003ef0 msgr2=0x7f16040047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:54.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.578+0000 7f162ac00640 1 --2- 172.21.15.143:0/4249972021 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1604003ef0 0x7f16040047c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f160c00c270 tx=0x7f16040057f0 comp rx=0 tx=0).stop 2024-01-16T15:00:54.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.578+0000 7f1628c00640 1 -- 172.21.15.143:0/4249972021 reap_dead start 2024-01-16T15:00:54.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.578+0000 7f162ac00640 1 -- 172.21.15.143:0/4249972021 shutdown_connections 2024-01-16T15:00:54.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.578+0000 7f162ac00640 1 -- 172.21.15.143:0/4249972021 >> 172.21.15.143:0/4249972021 conn(0x7f16240b2e40 msgr2=0x7f16240b3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:54.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.578+0000 7f162ac00640 1 -- 172.21.15.143:0/4249972021 shutdown_connections 2024-01-16T15:00:54.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.578+0000 7f162ac00640 1 -- 172.21.15.143:0/4249972021 wait complete. 2024-01-16T15:00:54.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.580+0000 7f162ac00640 1 Processor -- start 2024-01-16T15:00:54.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.580+0000 7f162ac00640 1 -- start start 2024-01-16T15:00:54.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.581+0000 7f162ac00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1604003ef0 0x7f16240b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:54.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.581+0000 7f162ac00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f162413f7d0 0x7f162415f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:54.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.581+0000 7f162ac00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f162415f7e0 0x7f16241f6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:54.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.581+0000 7f162ac00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1624153140 con 0x7f162415f7e0 2024-01-16T15:00:54.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.581+0000 7f162ac00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16240b48e0 con 0x7f1604003ef0 2024-01-16T15:00:54.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.581+0000 7f162ac00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1624152e60 con 0x7f162413f7d0 2024-01-16T15:00:54.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.581+0000 7f1628c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f162415f7e0 0x7f16241f6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:54.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.581+0000 7f1628c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f162415f7e0 0x7f16241f6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60006/0 (socket says 172.21.15.143:60006) 2024-01-16T15:00:54.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.581+0000 7f1628c00640 1 -- 172.21.15.143:0/4109213586 learned_addr learned my addr 172.21.15.143:0/4109213586 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:54.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.582+0000 7f1623e00640 1 --2- 172.21.15.143:0/4109213586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1604003ef0 0x7f16240b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:54.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.582+0000 7f161be00640 1 --2- 172.21.15.143:0/4109213586 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f162413f7d0 0x7f162415f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:54.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.582+0000 7f1628c00640 1 -- 172.21.15.143:0/4109213586 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f162413f7d0 msgr2=0x7f162415f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:54.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.582+0000 7f1628c00640 1 --2- 172.21.15.143:0/4109213586 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f162413f7d0 0x7f162415f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:54.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.582+0000 7f1628c00640 1 -- 172.21.15.143:0/4109213586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1604003ef0 msgr2=0x7f16240b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:54.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.582+0000 7f1628c00640 1 --2- 172.21.15.143:0/4109213586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1604003ef0 0x7f16240b43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:54.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.582+0000 7f1628c00640 1 -- 172.21.15.143:0/4109213586 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16241558b0 con 0x7f162415f7e0 2024-01-16T15:00:54.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.582+0000 7f1628c00640 1 --2- 172.21.15.143:0/4109213586 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f162415f7e0 0x7f16241f6360 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1624140b90 tx=0x7f161c0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:54.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.582+0000 7f1620c00640 1 -- 172.21.15.143:0/4109213586 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161c00e070 con 0x7f162415f7e0 2024-01-16T15:00:54.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.582+0000 7f1620c00640 1 -- 172.21.15.143:0/4109213586 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161c011e10 con 0x7f162415f7e0 2024-01-16T15:00:54.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.582+0000 7f162ac00640 1 -- 172.21.15.143:0/4109213586 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1624155b90 con 0x7f162415f7e0 2024-01-16T15:00:54.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.582+0000 7f1620c00640 1 -- 172.21.15.143:0/4109213586 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161c003990 con 0x7f162415f7e0 2024-01-16T15:00:54.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.582+0000 7f162ac00640 1 -- 172.21.15.143:0/4109213586 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1624152a80 con 0x7f162415f7e0 2024-01-16T15:00:54.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.584+0000 7f162ac00640 1 -- 172.21.15.143:0/4109213586 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1624155d80 con 0x7f162415f7e0 2024-01-16T15:00:54.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.584+0000 7f1620c00640 1 -- 172.21.15.143:0/4109213586 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f161c003af0 con 0x7f162415f7e0 2024-01-16T15:00:54.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.586+0000 7f1620c00640 1 --2- 172.21.15.143:0/4109213586 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f15fc06b2f0 0x7f15fc06d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:54.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.586+0000 7f1620c00640 1 -- 172.21.15.143:0/4109213586 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f161c08a1c0 con 0x7f162415f7e0 2024-01-16T15:00:54.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.586+0000 7f1623e00640 1 --2- 172.21.15.143:0/4109213586 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f15fc06b2f0 0x7f15fc06d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:54.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.587+0000 7f1623e00640 1 --2- 172.21.15.143:0/4109213586 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f15fc06b2f0 0x7f15fc06d7b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1614009aa0 tx=0x7f1614008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:54.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.590+0000 7f1620c00640 1 -- 172.21.15.143:0/4109213586 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f161c054f10 con 0x7f162415f7e0 2024-01-16T15:00:54.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.792+0000 7f162ac00640 1 -- 172.21.15.143:0/4109213586 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f162404e8b0 con 0x7f162415f7e0 2024-01-16T15:00:54.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.793+0000 7f1620c00640 1 -- 172.21.15.143:0/4109213586 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f161c058530 con 0x7f162415f7e0 2024-01-16T15:00:54.774 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2024-01-16T15:00:54.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.795+0000 7f162ac00640 1 -- 172.21.15.143:0/4109213586 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f15fc06b2f0 msgr2=0x7f15fc06d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:54.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.795+0000 7f162ac00640 1 --2- 172.21.15.143:0/4109213586 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f15fc06b2f0 0x7f15fc06d7b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1614009aa0 tx=0x7f1614008040 comp rx=0 tx=0).stop 2024-01-16T15:00:54.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.795+0000 7f162ac00640 1 -- 172.21.15.143:0/4109213586 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f162415f7e0 msgr2=0x7f16241f6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:54.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.795+0000 7f162ac00640 1 --2- 172.21.15.143:0/4109213586 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f162415f7e0 0x7f16241f6360 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1624140b90 tx=0x7f161c0115a0 comp rx=0 tx=0).stop 2024-01-16T15:00:54.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.796+0000 7f162ac00640 1 -- 172.21.15.143:0/4109213586 shutdown_connections 2024-01-16T15:00:54.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.796+0000 7f162ac00640 1 --2- 172.21.15.143:0/4109213586 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f15fc06b2f0 0x7f15fc06d7b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:54.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.796+0000 7f162ac00640 1 --2- 172.21.15.143:0/4109213586 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f162415f7e0 0x7f16241f6360 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:54.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.796+0000 7f162ac00640 1 --2- 172.21.15.143:0/4109213586 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f162413f7d0 0x7f162415f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:54.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.796+0000 7f162ac00640 1 --2- 172.21.15.143:0/4109213586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1604003ef0 0x7f16240b43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:54.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.796+0000 7f162ac00640 1 -- 172.21.15.143:0/4109213586 >> 172.21.15.143:0/4109213586 conn(0x7f16240b2e40 msgr2=0x7f162415d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:54.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.796+0000 7f162ac00640 1 -- 172.21.15.143:0/4109213586 shutdown_connections 2024-01-16T15:00:54.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:54.796+0000 7f162ac00640 1 -- 172.21.15.143:0/4109213586 wait complete. 2024-01-16T15:00:54.795 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-16T15:00:55.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.023+0000 7fcb73200640 1 Processor -- start 2024-01-16T15:00:55.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.024+0000 7fcb73200640 1 -- start start 2024-01-16T15:00:55.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.025+0000 7fcb73200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb740cd8a0 0x7fcb740c2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:55.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.025+0000 7fcb73200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb740cec30 con 0x7fcb74161810 2024-01-16T15:00:55.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.025+0000 7fcb73200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb740ce950 con 0x7fcb740c25d0 2024-01-16T15:00:55.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.025+0000 7fcb73200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb740c2a60 con 0x7fcb740cd8a0 2024-01-16T15:00:55.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.025+0000 7fcb71e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fcb74161810 0x7fcb740cdf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43380/0 (socket says 172.21.15.143:43380) 2024-01-16T15:00:55.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.025+0000 7fcb71e00640 1 -- 172.21.15.143:0/3261145349 learned_addr learned my addr 172.21.15.143:0/3261145349 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:55.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.026+0000 7fcb69200640 1 --2- 172.21.15.143:0/3261145349 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb740cd8a0 0x7fcb740c2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:55.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.026+0000 7fcb71400640 1 -- 172.21.15.143:0/3261145349 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2466663090 0 0) 0x7fcb740cec30 con 0x7fcb74161810 2024-01-16T15:00:55.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.026+0000 7fcb71400640 1 -- 172.21.15.143:0/3261145349 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb54003520 con 0x7fcb74161810 2024-01-16T15:00:55.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.026+0000 7fcb71400640 1 -- 172.21.15.143:0/3261145349 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3415990904 0 0) 0x7fcb54003520 con 0x7fcb74161810 2024-01-16T15:00:55.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.026+0000 7fcb71400640 1 -- 172.21.15.143:0/3261145349 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb740cd8a0 msgr2=0x7fcb740c2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:55.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.026+0000 7fcb71400640 1 --2- 172.21.15.143:0/3261145349 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb740cd8a0 0x7fcb740c2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:55.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.026+0000 7fcb71400640 1 -- 172.21.15.143:0/3261145349 >> v1:172.21.15.204:6789/0 conn(0x7fcb740c25d0 legacy=0x7fcb74165380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:55.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.026+0000 7fcb71400640 1 -- 172.21.15.143:0/3261145349 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb740ccdc0 con 0x7fcb74161810 2024-01-16T15:00:55.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.026+0000 7fcb71400640 1 -- 172.21.15.143:0/3261145349 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7fcb64001670 con 0x7fcb74161810 2024-01-16T15:00:55.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.026+0000 7fcb71400640 1 -- 172.21.15.143:0/3261145349 >> v1:172.21.15.143:6789/0 conn(0x7fcb74161810 legacy=0x7fcb740cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:55.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.026+0000 7fcb71400640 1 --2- 172.21.15.143:0/3261145349 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb740cd8a0 0x7fcb540039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:55.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.026+0000 7fcb71400640 1 --2- 172.21.15.143:0/3261145349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb54003fd0 0x7fcb540047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:55.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.026+0000 7fcb71400640 1 --2- 172.21.15.143:0/3261145349 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb54004d00 0x7fcb54005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:55.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.026+0000 7fcb71400640 1 -- 172.21.15.143:0/3261145349 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb740c2a60 con 0x7fcb54004d00 2024-01-16T15:00:55.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.026+0000 7fcb71400640 1 -- 172.21.15.143:0/3261145349 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5400bbc0 con 0x7fcb54003fd0 2024-01-16T15:00:55.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.026+0000 7fcb71400640 1 -- 172.21.15.143:0/3261145349 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5400bd80 con 0x7fcb740cd8a0 2024-01-16T15:00:55.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.027+0000 7fcb72800640 1 --2- 172.21.15.143:0/3261145349 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb54004d00 0x7fcb54005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:55.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.027+0000 7fcb71e00640 1 --2- 172.21.15.143:0/3261145349 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb740cd8a0 0x7fcb540039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:55.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.027+0000 7fcb69200640 1 --2- 172.21.15.143:0/3261145349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb54003fd0 0x7fcb540047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:55.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.027+0000 7fcb72800640 1 -- 172.21.15.143:0/3261145349 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb740cd8a0 msgr2=0x7fcb540039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:55.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.027+0000 7fcb72800640 1 --2- 172.21.15.143:0/3261145349 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb740cd8a0 0x7fcb540039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:55.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.027+0000 7fcb72800640 1 -- 172.21.15.143:0/3261145349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb54003fd0 msgr2=0x7fcb540047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:55.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.027+0000 7fcb72800640 1 --2- 172.21.15.143:0/3261145349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb54003fd0 0x7fcb540047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:55.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.027+0000 7fcb72800640 1 -- 172.21.15.143:0/3261145349 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb54003520 con 0x7fcb54004d00 2024-01-16T15:00:55.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.028+0000 7fcb72800640 1 --2- 172.21.15.143:0/3261145349 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb54004d00 0x7fcb54005180 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c001e50 tx=0x7fcb540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13427ea8980cc2c server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:55.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.028+0000 7fcb71400640 1 -- 172.21.15.143:0/3261145349 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c003b70 con 0x7fcb54004d00 2024-01-16T15:00:55.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.028+0000 7fcb71400640 1 -- 172.21.15.143:0/3261145349 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb6c003cd0 con 0x7fcb54004d00 2024-01-16T15:00:55.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.028+0000 7fcb71400640 1 -- 172.21.15.143:0/3261145349 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c005310 con 0x7fcb54004d00 2024-01-16T15:00:55.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.029+0000 7fcb73200640 1 -- 172.21.15.143:0/3261145349 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb54004d00 msgr2=0x7fcb54005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:55.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.029+0000 7fcb73200640 1 --2- 172.21.15.143:0/3261145349 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb54004d00 0x7fcb54005180 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c001e50 tx=0x7fcb540057f0 comp rx=0 tx=0).stop 2024-01-16T15:00:55.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.029+0000 7fcb72800640 1 -- 172.21.15.143:0/3261145349 reap_dead start 2024-01-16T15:00:55.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.029+0000 7fcb73200640 1 -- 172.21.15.143:0/3261145349 shutdown_connections 2024-01-16T15:00:55.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.029+0000 7fcb73200640 1 -- 172.21.15.143:0/3261145349 >> 172.21.15.143:0/3261145349 conn(0x7fcb740cbb40 msgr2=0x7fcb740cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:55.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.029+0000 7fcb73200640 1 -- 172.21.15.143:0/3261145349 shutdown_connections 2024-01-16T15:00:55.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.029+0000 7fcb73200640 1 -- 172.21.15.143:0/3261145349 wait complete. 2024-01-16T15:00:55.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.031+0000 7fcb73200640 1 Processor -- start 2024-01-16T15:00:55.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.031+0000 7fcb73200640 1 -- start start 2024-01-16T15:00:55.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.032+0000 7fcb73200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb54004d00 0x7fcb7415c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:55.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.032+0000 7fcb73200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb740c4d70 0x7fcb740c51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:55.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.032+0000 7fcb73200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb741f7470 0x7fcb741f7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:55.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.032+0000 7fcb73200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb740c3260 con 0x7fcb741f7470 2024-01-16T15:00:55.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.032+0000 7fcb73200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb740c5730 con 0x7fcb740c4d70 2024-01-16T15:00:55.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.032+0000 7fcb73200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb741f7e60 con 0x7fcb54004d00 2024-01-16T15:00:55.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.032+0000 7fcb72800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb741f7470 0x7fcb741f7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:55.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.032+0000 7fcb72800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb741f7470 0x7fcb741f7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47902/0 (socket says 172.21.15.143:47902) 2024-01-16T15:00:55.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.032+0000 7fcb72800640 1 -- 172.21.15.143:0/3435814622 learned_addr learned my addr 172.21.15.143:0/3435814622 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:55.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.032+0000 7fcb71e00640 1 --2- 172.21.15.143:0/3435814622 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb54004d00 0x7fcb7415c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:55.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.032+0000 7fcb69200640 1 --2- 172.21.15.143:0/3435814622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb740c4d70 0x7fcb740c51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:55.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.032+0000 7fcb72800640 1 -- 172.21.15.143:0/3435814622 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb54004d00 msgr2=0x7fcb7415c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:55.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.032+0000 7fcb72800640 1 --2- 172.21.15.143:0/3435814622 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb54004d00 0x7fcb7415c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:55.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.032+0000 7fcb72800640 1 -- 172.21.15.143:0/3435814622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb740c4d70 msgr2=0x7fcb740c51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:55.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.032+0000 7fcb72800640 1 --2- 172.21.15.143:0/3435814622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb740c4d70 0x7fcb740c51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:55.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.032+0000 7fcb72800640 1 -- 172.21.15.143:0/3435814622 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb74164ce0 con 0x7fcb741f7470 2024-01-16T15:00:55.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.033+0000 7fcb69200640 1 --2- 172.21.15.143:0/3435814622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb740c4d70 0x7fcb740c51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:55.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.033+0000 7fcb72800640 1 --2- 172.21.15.143:0/3435814622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb741f7470 0x7fcb741f7920 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c02c060 tx=0x7fcb6c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:55.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.033+0000 7fcb6aa00640 1 -- 172.21.15.143:0/3435814622 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c03e070 con 0x7fcb741f7470 2024-01-16T15:00:55.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.033+0000 7fcb6aa00640 1 -- 172.21.15.143:0/3435814622 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb6c003e40 con 0x7fcb741f7470 2024-01-16T15:00:55.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.033+0000 7fcb6aa00640 1 -- 172.21.15.143:0/3435814622 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c006d90 con 0x7fcb741f7470 2024-01-16T15:00:55.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.033+0000 7fcb73200640 1 -- 172.21.15.143:0/3435814622 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb741f8080 con 0x7fcb741f7470 2024-01-16T15:00:55.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.033+0000 7fcb73200640 1 -- 172.21.15.143:0/3435814622 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb740c2ba0 con 0x7fcb741f7470 2024-01-16T15:00:55.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.034+0000 7fcb73200640 1 -- 172.21.15.143:0/3435814622 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb7404e840 con 0x7fcb741f7470 2024-01-16T15:00:55.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.035+0000 7fcb6aa00640 1 -- 172.21.15.143:0/3435814622 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fcb6c041070 con 0x7fcb741f7470 2024-01-16T15:00:55.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.036+0000 7fcb6aa00640 1 --2- 172.21.15.143:0/3435814622 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fcb4c06b2c0 0x7fcb4c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:55.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.036+0000 7fcb6aa00640 1 -- 172.21.15.143:0/3435814622 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fcb6c0aab10 con 0x7fcb741f7470 2024-01-16T15:00:55.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.036+0000 7fcb71e00640 1 --2- 172.21.15.143:0/3435814622 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fcb4c06b2c0 0x7fcb4c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:55.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.037+0000 7fcb71e00640 1 --2- 172.21.15.143:0/3435814622 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fcb4c06b2c0 0x7fcb4c06d780 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcb64001eb0 tx=0x7fcb64001d60 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:55.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.040+0000 7fcb6aa00640 1 -- 172.21.15.143:0/3435814622 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcb6c0759e0 con 0x7fcb741f7470 2024-01-16T15:00:55.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.243+0000 7fcb73200640 1 -- 172.21.15.143:0/3435814622 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcb7406d200 con 0x7fcb741f7470 2024-01-16T15:00:55.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.244+0000 7fcb6aa00640 1 -- 172.21.15.143:0/3435814622 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fcb6c079000 con 0x7fcb741f7470 2024-01-16T15:00:55.225 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2024-01-16T15:00:55.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.247+0000 7fcb73200640 1 -- 172.21.15.143:0/3435814622 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fcb4c06b2c0 msgr2=0x7fcb4c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:55.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.247+0000 7fcb73200640 1 --2- 172.21.15.143:0/3435814622 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fcb4c06b2c0 0x7fcb4c06d780 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcb64001eb0 tx=0x7fcb64001d60 comp rx=0 tx=0).stop 2024-01-16T15:00:55.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.247+0000 7fcb73200640 1 -- 172.21.15.143:0/3435814622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb741f7470 msgr2=0x7fcb741f7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:55.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.247+0000 7fcb73200640 1 --2- 172.21.15.143:0/3435814622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb741f7470 0x7fcb741f7920 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c02c060 tx=0x7fcb6c0014a0 comp rx=0 tx=0).stop 2024-01-16T15:00:55.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.247+0000 7fcb73200640 1 -- 172.21.15.143:0/3435814622 shutdown_connections 2024-01-16T15:00:55.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.247+0000 7fcb73200640 1 --2- 172.21.15.143:0/3435814622 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fcb4c06b2c0 0x7fcb4c06d780 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:55.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.247+0000 7fcb73200640 1 --2- 172.21.15.143:0/3435814622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb741f7470 0x7fcb741f7920 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:55.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.247+0000 7fcb73200640 1 --2- 172.21.15.143:0/3435814622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb740c4d70 0x7fcb740c51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:55.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.247+0000 7fcb73200640 1 --2- 172.21.15.143:0/3435814622 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb54004d00 0x7fcb7415c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:55.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.247+0000 7fcb73200640 1 -- 172.21.15.143:0/3435814622 >> 172.21.15.143:0/3435814622 conn(0x7fcb740cbb40 msgr2=0x7fcb740cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:55.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.247+0000 7fcb73200640 1 -- 172.21.15.143:0/3435814622 shutdown_connections 2024-01-16T15:00:55.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.247+0000 7fcb73200640 1 -- 172.21.15.143:0/3435814622 wait complete. 2024-01-16T15:00:55.246 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-01-16T15:00:55.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:55.246 DEBUG:teuthology.orchestra.run.smithi204:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-01-16T15:00:55.353 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi204.front.sepia.ceph.com 2024-01-16T15:00:55.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:00:55.354 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-01-16T15:00:55.402 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-16T15:00:55.402 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-16T15:00:55.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.632+0000 7f4bc8a00640 1 Processor -- start 2024-01-16T15:00:55.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.633+0000 7f4bc8a00640 1 -- start start 2024-01-16T15:00:55.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.634+0000 7f4bc8a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4bc4144b30 0x7f4bc415f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:55.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.634+0000 7f4bc8a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4bc415c410 0x7f4bc40b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:55.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.634+0000 7f4bc8a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc415bf80 con 0x7f4bc40b1de0 2024-01-16T15:00:55.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.634+0000 7f4bc8a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bc40b1560 con 0x7f4bc4144b30 2024-01-16T15:00:55.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.634+0000 7f4bc8a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bc40b0fb0 con 0x7f4bc415c410 2024-01-16T15:00:55.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.635+0000 7f4bc3400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4bc4144b30 0x7f4bc415f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:55.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.635+0000 7f4bc3400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4bc4144b30 0x7f4bc415f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47908/0 (socket says 172.21.15.143:47908) 2024-01-16T15:00:55.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.635+0000 7f4bc3400640 1 -- 172.21.15.143:0/862553892 learned_addr learned my addr 172.21.15.143:0/862553892 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:55.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.635+0000 7f4bba800640 1 --2- 172.21.15.143:0/862553892 >> v2:172.21.15.204:3300/0 conn(0x7f4bc415c410 0x7f4bc40b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:55.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.635+0000 7f4bc3400640 1 -- 172.21.15.143:0/862553892 >> v2:172.21.15.204:3300/0 conn(0x7f4bc415c410 msgr2=0x7f4bc40b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:55.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.635+0000 7f4bc3400640 1 --2- 172.21.15.143:0/862553892 >> v2:172.21.15.204:3300/0 conn(0x7f4bc415c410 0x7f4bc40b0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:55.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.635+0000 7f4bc3400640 1 -- 172.21.15.143:0/862553892 >> v1:172.21.15.204:6789/0 conn(0x7f4bc40b1de0 legacy=0x7f4bc406c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:55.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.635+0000 7f4bc3400640 1 -- 172.21.15.143:0/862553892 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc40b1280 con 0x7f4bc4144b30 2024-01-16T15:00:55.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.636+0000 7f4bc3400640 1 --2- 172.21.15.143:0/862553892 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bc4144b30 0x7f4bc415f920 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4bc40cc7a0 tx=0x7f4bb402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5796114d1c73a88 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:55.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.637+0000 7f4bc2a00640 1 -- 172.21.15.143:0/862553892 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4031870 con 0x7f4bc4144b30 2024-01-16T15:00:55.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.637+0000 7f4bc2a00640 1 -- 172.21.15.143:0/862553892 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb40319d0 con 0x7f4bc4144b30 2024-01-16T15:00:55.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.637+0000 7f4bc2a00640 1 -- 172.21.15.143:0/862553892 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4031d60 con 0x7f4bc4144b30 2024-01-16T15:00:55.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.637+0000 7f4bc8a00640 1 -- 172.21.15.143:0/862553892 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bc4144b30 msgr2=0x7f4bc415f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:55.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.637+0000 7f4bc8a00640 1 --2- 172.21.15.143:0/862553892 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bc4144b30 0x7f4bc415f920 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4bc40cc7a0 tx=0x7f4bb402b910 comp rx=0 tx=0).stop 2024-01-16T15:00:55.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.637+0000 7f4bc8a00640 1 -- 172.21.15.143:0/862553892 shutdown_connections 2024-01-16T15:00:55.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.637+0000 7f4bc8a00640 1 --2- 172.21.15.143:0/862553892 >> v2:172.21.15.204:3300/0 conn(0x7f4bc415c410 0x7f4bc40b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:55.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.637+0000 7f4bc8a00640 1 --2- 172.21.15.143:0/862553892 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bc4144b30 0x7f4bc415f920 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:55.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.637+0000 7f4bc8a00640 1 -- 172.21.15.143:0/862553892 >> 172.21.15.143:0/862553892 conn(0x7f4bc40cb830 msgr2=0x7f4bc40cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:55.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.637+0000 7f4bc8a00640 1 -- 172.21.15.143:0/862553892 shutdown_connections 2024-01-16T15:00:55.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.637+0000 7f4bc8a00640 1 -- 172.21.15.143:0/862553892 wait complete. 2024-01-16T15:00:55.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.639+0000 7f4bc8a00640 1 Processor -- start 2024-01-16T15:00:55.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.639+0000 7f4bc8a00640 1 -- start start 2024-01-16T15:00:55.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.640+0000 7f4bc8a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bc40b1de0 0x7f4bc415a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:55.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.640+0000 7f4bc8a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4bc4144b30 0x7f4bc40c7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:55.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.640+0000 7f4bc8a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4bc415c410 0x7f4bc40cd630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:55.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.640+0000 7f4bc8a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc40c8b90 con 0x7f4bc40b1de0 2024-01-16T15:00:55.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.640+0000 7f4bc8a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc40c7f10 con 0x7f4bc4144b30 2024-01-16T15:00:55.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.640+0000 7f4bc8a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc40c81f0 con 0x7f4bc415c410 2024-01-16T15:00:55.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.640+0000 7f4bc3400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bc40b1de0 0x7f4bc415a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:55.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.640+0000 7f4bc3400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bc40b1de0 0x7f4bc415a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47924/0 (socket says 172.21.15.143:47924) 2024-01-16T15:00:55.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.640+0000 7f4bc3400640 1 -- 172.21.15.143:0/3653327640 learned_addr learned my addr 172.21.15.143:0/3653327640 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:55.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.641+0000 7f4bc3400640 1 -- 172.21.15.143:0/3653327640 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4bc415c410 msgr2=0x7f4bc40cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:55.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.641+0000 7f4bc3400640 1 --2- 172.21.15.143:0/3653327640 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4bc415c410 0x7f4bc40cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:55.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.641+0000 7f4bc3400640 1 -- 172.21.15.143:0/3653327640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4bc4144b30 msgr2=0x7f4bc40c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:55.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.641+0000 7f4bc3400640 1 --2- 172.21.15.143:0/3653327640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4bc4144b30 0x7f4bc40c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:55.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.641+0000 7f4bc3400640 1 -- 172.21.15.143:0/3653327640 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb40094d0 con 0x7f4bc40b1de0 2024-01-16T15:00:55.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.641+0000 7f4bc3400640 1 --2- 172.21.15.143:0/3653327640 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bc40b1de0 0x7f4bc415a820 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4000c00 tx=0x7f4bb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:55.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.641+0000 7f4bbbe00640 1 -- 172.21.15.143:0/3653327640 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb403ad10 con 0x7f4bc40b1de0 2024-01-16T15:00:55.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.641+0000 7f4bbbe00640 1 -- 172.21.15.143:0/3653327640 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb403ae90 con 0x7f4bc40b1de0 2024-01-16T15:00:55.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.641+0000 7f4bc8a00640 1 -- 172.21.15.143:0/3653327640 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc40c84d0 con 0x7f4bc40b1de0 2024-01-16T15:00:55.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.641+0000 7f4bbbe00640 1 -- 172.21.15.143:0/3653327640 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4044630 con 0x7f4bc40b1de0 2024-01-16T15:00:55.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.641+0000 7f4bc8a00640 1 -- 172.21.15.143:0/3653327640 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc415ad60 con 0x7f4bc40b1de0 2024-01-16T15:00:55.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.642+0000 7f4bc8a00640 1 -- 172.21.15.143:0/3653327640 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bc404e840 con 0x7f4bc40b1de0 2024-01-16T15:00:55.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.643+0000 7f4bbbe00640 1 -- 172.21.15.143:0/3653327640 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f4bb404d450 con 0x7f4bc40b1de0 2024-01-16T15:00:55.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.644+0000 7f4bbbe00640 1 --2- 172.21.15.143:0/3653327640 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f4b9c06b2c0 0x7f4b9c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:55.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.644+0000 7f4bbbe00640 1 -- 172.21.15.143:0/3653327640 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f4bb402d040 con 0x7f4bc40b1de0 2024-01-16T15:00:55.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.644+0000 7f4bba800640 1 --2- 172.21.15.143:0/3653327640 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f4b9c06b2c0 0x7f4b9c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:55.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.645+0000 7f4bba800640 1 --2- 172.21.15.143:0/3653327640 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f4b9c06b2c0 0x7f4b9c06d780 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4140730 tx=0x7f4bac008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:55.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.648+0000 7f4bbbe00640 1 -- 172.21.15.143:0/3653327640 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4bb40771b0 con 0x7f4bc40b1de0 2024-01-16T15:00:55.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.862+0000 7f4bc8a00640 1 -- 172.21.15.143:0/3653327640 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4bc406d860 con 0x7f4bc40b1de0 2024-01-16T15:00:55.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.863+0000 7f4bbbe00640 1 -- 172.21.15.143:0/3653327640 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4916 (secure 0 0 0) 0x7f4bb407a7d0 con 0x7f4bc40b1de0 2024-01-16T15:00:55.844 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:00:55.844 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3599169408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3599169408},{"type":"v1","addr":"172.21.15.204:6834","nonce":3599169408}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/859082411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":859082411},{"type":"v1","addr":"172.21.15.204:6835","nonce":859082411}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6836/1564738611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1564738611},{"type":"v1","addr":"172.21.15.143:6836","nonce":1564738611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-16T15:00:03.050082+0000","modified":"2024-01-16T15:00:06.109664+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/3120706283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6835","nonce":3120706283},{"type":"v1","addr":"172.21.15.143:6837","nonce":3120706283}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-16T15:00:55.845 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 8 2024-01-16T15:00:55.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.866+0000 7f4bc8a00640 1 -- 172.21.15.143:0/3653327640 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f4b9c06b2c0 msgr2=0x7f4b9c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:55.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.866+0000 7f4bc8a00640 1 --2- 172.21.15.143:0/3653327640 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f4b9c06b2c0 0x7f4b9c06d780 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4140730 tx=0x7f4bac008040 comp rx=0 tx=0).stop 2024-01-16T15:00:55.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.866+0000 7f4bc8a00640 1 -- 172.21.15.143:0/3653327640 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bc40b1de0 msgr2=0x7f4bc415a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:55.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.866+0000 7f4bc8a00640 1 --2- 172.21.15.143:0/3653327640 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bc40b1de0 0x7f4bc415a820 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4000c00 tx=0x7f4bb40046c0 comp rx=0 tx=0).stop 2024-01-16T15:00:55.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.866+0000 7f4bc8a00640 1 -- 172.21.15.143:0/3653327640 shutdown_connections 2024-01-16T15:00:55.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.866+0000 7f4bc8a00640 1 --2- 172.21.15.143:0/3653327640 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f4b9c06b2c0 0x7f4b9c06d780 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:55.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.866+0000 7f4bc8a00640 1 --2- 172.21.15.143:0/3653327640 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4bc415c410 0x7f4bc40cd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:55.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.866+0000 7f4bc8a00640 1 --2- 172.21.15.143:0/3653327640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4bc4144b30 0x7f4bc40c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:55.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.866+0000 7f4bc8a00640 1 --2- 172.21.15.143:0/3653327640 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bc40b1de0 0x7f4bc415a820 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:55.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.866+0000 7f4bc8a00640 1 -- 172.21.15.143:0/3653327640 >> 172.21.15.143:0/3653327640 conn(0x7f4bc40cb830 msgr2=0x7f4bc4142f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:55.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.866+0000 7f4bc8a00640 1 -- 172.21.15.143:0/3653327640 shutdown_connections 2024-01-16T15:00:55.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:55.866+0000 7f4bc8a00640 1 -- 172.21.15.143:0/3653327640 wait complete. 2024-01-16T15:00:55.864 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T15:00:56.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.090+0000 7f8486400640 1 Processor -- start 2024-01-16T15:00:56.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.090+0000 7f8486400640 1 -- start start 2024-01-16T15:00:56.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.091+0000 7f8486400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f848015c410 0x7f848015f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:56.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.091+0000 7f8486400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84800b1de0 0x7f84800b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:56.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.091+0000 7f8486400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f848015bf80 con 0x7f8480144b30 2024-01-16T15:00:56.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.091+0000 7f8486400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84800b27b0 con 0x7f848015c410 2024-01-16T15:00:56.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.091+0000 7f8486400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84800b1560 con 0x7f84800b1de0 2024-01-16T15:00:56.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.092+0000 7f8485000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8480144b30 0x7f84800b0a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43386/0 (socket says 172.21.15.143:43386) 2024-01-16T15:00:56.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.092+0000 7f8485000640 1 -- 172.21.15.143:0/925522336 learned_addr learned my addr 172.21.15.143:0/925522336 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:56.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.092+0000 7f847fe00640 1 -- 172.21.15.143:0/925522336 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3312679498 0 0) 0x7f848015bf80 con 0x7f8480144b30 2024-01-16T15:00:56.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.092+0000 7f847fe00640 1 -- 172.21.15.143:0/925522336 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8460003520 con 0x7f8480144b30 2024-01-16T15:00:56.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.092+0000 7f8477c00640 1 --2- 172.21.15.143:0/925522336 >> v2:172.21.15.204:3300/0 conn(0x7f848015c410 0x7f848015f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:56.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.092+0000 7f8485a00640 1 --2- 172.21.15.143:0/925522336 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84800b1de0 0x7f84800b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:56.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.093+0000 7f847fe00640 1 -- 172.21.15.143:0/925522336 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2633879287 0 0) 0x7f8460003520 con 0x7f8480144b30 2024-01-16T15:00:56.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.093+0000 7f847fe00640 1 -- 172.21.15.143:0/925522336 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84800b1de0 msgr2=0x7f84800b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:56.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.093+0000 7f847fe00640 1 --2- 172.21.15.143:0/925522336 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84800b1de0 0x7f84800b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:56.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.093+0000 7f847fe00640 1 -- 172.21.15.143:0/925522336 >> v2:172.21.15.204:3300/0 conn(0x7f848015c410 msgr2=0x7f848015f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:56.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.093+0000 7f847fe00640 1 --2- 172.21.15.143:0/925522336 >> v2:172.21.15.204:3300/0 conn(0x7f848015c410 0x7f848015f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:56.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.093+0000 7f847fe00640 1 -- 172.21.15.143:0/925522336 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84800b1280 con 0x7f8480144b30 2024-01-16T15:00:56.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.093+0000 7f847fe00640 1 -- 172.21.15.143:0/925522336 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7f84700019f0 con 0x7f8480144b30 2024-01-16T15:00:56.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.093+0000 7f847fe00640 1 -- 172.21.15.143:0/925522336 >> v1:172.21.15.143:6789/0 conn(0x7f8480144b30 legacy=0x7f84800b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:56.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.093+0000 7f847fe00640 1 --2- 172.21.15.143:0/925522336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8460003970 0x7f8460003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:56.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.093+0000 7f847fe00640 1 --2- 172.21.15.143:0/925522336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84600047c0 0x7f8460004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:56.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.093+0000 7f847fe00640 1 --2- 172.21.15.143:0/925522336 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84800b1de0 0x7f8460005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:56.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.093+0000 7f847fe00640 1 -- 172.21.15.143:0/925522336 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84800b27b0 con 0x7f84600047c0 2024-01-16T15:00:56.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.093+0000 7f847fe00640 1 -- 172.21.15.143:0/925522336 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84800b1560 con 0x7f8460003970 2024-01-16T15:00:56.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.093+0000 7f847fe00640 1 -- 172.21.15.143:0/925522336 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f846000b980 con 0x7f84800b1de0 2024-01-16T15:00:56.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.093+0000 7f8477c00640 1 --2- 172.21.15.143:0/925522336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84600047c0 0x7f8460004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:56.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.094+0000 7f8477c00640 1 -- 172.21.15.143:0/925522336 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84800b1de0 msgr2=0x7f8460005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:56.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.094+0000 7f8477c00640 1 --2- 172.21.15.143:0/925522336 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84800b1de0 0x7f8460005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:56.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.094+0000 7f8477c00640 1 -- 172.21.15.143:0/925522336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8460003970 msgr2=0x7f8460003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:56.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.094+0000 7f8477c00640 1 --2- 172.21.15.143:0/925522336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8460003970 0x7f8460003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:56.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.094+0000 7f8477c00640 1 -- 172.21.15.143:0/925522336 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8460003520 con 0x7f84600047c0 2024-01-16T15:00:56.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.095+0000 7f8477c00640 1 --2- 172.21.15.143:0/925522336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84600047c0 0x7f8460004c40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8468009ac0 tx=0x7f84600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b3cde474593daa server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:56.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.095+0000 7f847fe00640 1 -- 172.21.15.143:0/925522336 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846802c710 con 0x7f84600047c0 2024-01-16T15:00:56.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.095+0000 7f847fe00640 1 -- 172.21.15.143:0/925522336 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846802c870 con 0x7f84600047c0 2024-01-16T15:00:56.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.095+0000 7f847fe00640 1 -- 172.21.15.143:0/925522336 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846802cc00 con 0x7f84600047c0 2024-01-16T15:00:56.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.095+0000 7f8486400640 1 -- 172.21.15.143:0/925522336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84600047c0 msgr2=0x7f8460004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:56.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.095+0000 7f8486400640 1 --2- 172.21.15.143:0/925522336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84600047c0 0x7f8460004c40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8468009ac0 tx=0x7f84600057f0 comp rx=0 tx=0).stop 2024-01-16T15:00:56.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.095+0000 7f8485a00640 1 -- 172.21.15.143:0/925522336 reap_dead start 2024-01-16T15:00:56.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.096+0000 7f8486400640 1 -- 172.21.15.143:0/925522336 shutdown_connections 2024-01-16T15:00:56.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.096+0000 7f8486400640 1 -- 172.21.15.143:0/925522336 >> 172.21.15.143:0/925522336 conn(0x7f84800cb830 msgr2=0x7f84800cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:56.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.096+0000 7f8486400640 1 -- 172.21.15.143:0/925522336 shutdown_connections 2024-01-16T15:00:56.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.096+0000 7f8486400640 1 -- 172.21.15.143:0/925522336 wait complete. 2024-01-16T15:00:56.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.097+0000 7f8486400640 1 Processor -- start 2024-01-16T15:00:56.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.098+0000 7f8486400640 1 -- start start 2024-01-16T15:00:56.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.098+0000 7f8486400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84600047c0 0x7f84800cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:56.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.098+0000 7f8486400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8480143520 0x7f84800ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:56.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.099+0000 7f8486400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84801368c0 0x7f84800cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:56.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.099+0000 7f8486400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84801439b0 con 0x7f84801368c0 2024-01-16T15:00:56.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.099+0000 7f8486400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84800c9c40 con 0x7f8480143520 2024-01-16T15:00:56.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.099+0000 7f8486400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84800c9310 con 0x7f84600047c0 2024-01-16T15:00:56.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.099+0000 7f8485a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84801368c0 0x7f84800cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:56.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.099+0000 7f8485a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84801368c0 0x7f84800cd1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47954/0 (socket says 172.21.15.143:47954) 2024-01-16T15:00:56.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.099+0000 7f8485a00640 1 -- 172.21.15.143:0/2820731136 learned_addr learned my addr 172.21.15.143:0/2820731136 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:56.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.099+0000 7f8485000640 1 --2- 172.21.15.143:0/2820731136 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84600047c0 0x7f84800cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:56.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.099+0000 7f8485a00640 1 -- 172.21.15.143:0/2820731136 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84600047c0 msgr2=0x7f84800cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:56.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.099+0000 7f8485a00640 1 --2- 172.21.15.143:0/2820731136 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84600047c0 0x7f84800cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:56.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.099+0000 7f8477c00640 1 --2- 172.21.15.143:0/2820731136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8480143520 0x7f84800ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:56.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.099+0000 7f8485a00640 1 -- 172.21.15.143:0/2820731136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8480143520 msgr2=0x7f84800ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:56.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.099+0000 7f8485a00640 1 --2- 172.21.15.143:0/2820731136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8480143520 0x7f84800ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:56.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.099+0000 7f8485a00640 1 -- 172.21.15.143:0/2820731136 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84800c95f0 con 0x7f84801368c0 2024-01-16T15:00:56.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.099+0000 7f8477c00640 1 --2- 172.21.15.143:0/2820731136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8480143520 0x7f84800ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:56.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.100+0000 7f8485a00640 1 --2- 172.21.15.143:0/2820731136 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84801368c0 0x7f84800cd1d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f848006e680 tx=0x7f847800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:56.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.100+0000 7f847d600640 1 -- 172.21.15.143:0/2820731136 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847800c620 con 0x7f84801368c0 2024-01-16T15:00:56.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.100+0000 7f847d600640 1 -- 172.21.15.143:0/2820731136 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f847800c780 con 0x7f84801368c0 2024-01-16T15:00:56.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.100+0000 7f847d600640 1 -- 172.21.15.143:0/2820731136 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847800cb30 con 0x7f84801368c0 2024-01-16T15:00:56.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.100+0000 7f8486400640 1 -- 172.21.15.143:0/2820731136 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84800c98d0 con 0x7f84801368c0 2024-01-16T15:00:56.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.100+0000 7f8486400640 1 -- 172.21.15.143:0/2820731136 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f848013ea10 con 0x7f84801368c0 2024-01-16T15:00:56.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.101+0000 7f8486400640 1 -- 172.21.15.143:0/2820731136 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f848004e840 con 0x7f84801368c0 2024-01-16T15:00:56.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.102+0000 7f847d600640 1 -- 172.21.15.143:0/2820731136 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f847800cc90 con 0x7f84801368c0 2024-01-16T15:00:56.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.103+0000 7f847d600640 1 --2- 172.21.15.143:0/2820731136 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f845806b2c0 0x7f845806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:56.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.104+0000 7f847d600640 1 -- 172.21.15.143:0/2820731136 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f8478089530 con 0x7f84801368c0 2024-01-16T15:00:56.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.104+0000 7f8485000640 1 --2- 172.21.15.143:0/2820731136 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f845806b2c0 0x7f845806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:56.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.104+0000 7f8485000640 1 --2- 172.21.15.143:0/2820731136 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f845806b2c0 0x7f845806d780 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f84700013a0 tx=0x7f8470009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:56.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.108+0000 7f847d600640 1 -- 172.21.15.143:0/2820731136 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f84780541f0 con 0x7f84801368c0 2024-01-16T15:00:56.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.297+0000 7f8486400640 1 -- 172.21.15.143:0/2820731136 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84800c9d80 con 0x7f84801368c0 2024-01-16T15:00:56.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.298+0000 7f847d600640 1 -- 172.21.15.143:0/2820731136 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16814 (secure 0 0 0) 0x7f8478057810 con 0x7f84801368c0 2024-01-16T15:00:56.280 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:00:56.280 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":22,"fsid":"c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0","created":"2024-01-16T14:59:38.532601+0000","modified":"2024-01-16T15:00:45.626415+0000","last_up_change":"2024-01-16T14:59:50.983699+0000","last_in_change":"2024-01-16T14:59:42.056297+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:59:51.727720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T15:00:01.010472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T15:00:02.041228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f4c8891d-ab29-472a-907d-8446f20f7012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6801","nonce":1569293626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6805","nonce":1569293626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6814","nonce":1569293626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6809","nonce":1569293626}]},"public_addr":"172.21.15.143:6801/1569293626","cluster_addr":"172.21.15.143:6805/1569293626","heartbeat_back_addr":"172.21.15.143:6814/1569293626","heartbeat_front_addr":"172.21.15.143:6809/1569293626","state":["exists","up"]},{"osd":1,"uuid":"c4a2788d-8b89-4992-82b2-7f4b9debd509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6804","nonce":1276841168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6808","nonce":1276841168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6813","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6815","nonce":1276841168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6811","nonce":1276841168}]},"public_addr":"172.21.15.143:6804/1276841168","cluster_addr":"172.21.15.143:6808/1276841168","heartbeat_back_addr":"172.21.15.143:6815/1276841168","heartbeat_front_addr":"172.21.15.143:6811/1276841168","state":["exists","up"]},{"osd":2,"uuid":"64645c1d-31b2-4794-b0f2-7ccf8df30393","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6817","nonce":1217670170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6819","nonce":1217670170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6823","nonce":1217670170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6821","nonce":1217670170}]},"public_addr":"172.21.15.143:6817/1217670170","cluster_addr":"172.21.15.143:6819/1217670170","heartbeat_back_addr":"172.21.15.143:6823/1217670170","heartbeat_front_addr":"172.21.15.143:6821/1217670170","state":["exists","up"]},{"osd":3,"uuid":"73674129-a84e-4d31-8247-7d0179ae6df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6825","nonce":3414188012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6827","nonce":3414188012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6831","nonce":3414188012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6829","nonce":3414188012}]},"public_addr":"172.21.15.143:6825/3414188012","cluster_addr":"172.21.15.143:6827/3414188012","heartbeat_back_addr":"172.21.15.143:6831/3414188012","heartbeat_front_addr":"172.21.15.143:6829/3414188012","state":["exists","up"]},{"osd":4,"uuid":"cea28c9b-2f1c-4087-b415-9960789131d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6811","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6812","nonce":1070255716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6816","nonce":1070255716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6823","nonce":1070255716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6820","nonce":1070255716}]},"public_addr":"172.21.15.204:6812/1070255716","cluster_addr":"172.21.15.204:6816/1070255716","heartbeat_back_addr":"172.21.15.204:6823/1070255716","heartbeat_front_addr":"172.21.15.204:6820/1070255716","state":["exists","up"]},{"osd":5,"uuid":"33b63560-5574-4e78-983f-b398d67a2cb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6809","nonce":3605417837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6813","nonce":3605417837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6819","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6821","nonce":3605417837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6817","nonce":3605417837}]},"public_addr":"172.21.15.204:6809/3605417837","cluster_addr":"172.21.15.204:6813/3605417837","heartbeat_back_addr":"172.21.15.204:6821/3605417837","heartbeat_front_addr":"172.21.15.204:6817/3605417837","state":["exists","up"]},{"osd":6,"uuid":"df397427-8940-4eee-9c79-eece669d735d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6801","nonce":1085000534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6803","nonce":1085000534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6808","nonce":1085000534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6805","nonce":1085000534}]},"public_addr":"172.21.15.204:6801/1085000534","cluster_addr":"172.21.15.204:6803/1085000534","heartbeat_back_addr":"172.21.15.204:6808/1085000534","heartbeat_front_addr":"172.21.15.204:6805/1085000534","state":["exists","up"]},{"osd":7,"uuid":"a2e5a15c-8312-405a-ac02-f7e1fc9df2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6825","nonce":1337220609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6827","nonce":1337220609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6831","nonce":1337220609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6829","nonce":1337220609}]},"public_addr":"172.21.15.204:6825/1337220609","cluster_addr":"172.21.15.204:6827/1337220609","heartbeat_back_addr":"172.21.15.204:6831/1337220609","heartbeat_front_addr":"172.21.15.204:6829/1337220609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:49.984507+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.395306+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.632998+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.373627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.342920+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.438366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.331926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.406657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T15:00:56.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.301+0000 7f8486400640 1 -- 172.21.15.143:0/2820731136 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f845806b2c0 msgr2=0x7f845806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:56.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.301+0000 7f8486400640 1 --2- 172.21.15.143:0/2820731136 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f845806b2c0 0x7f845806d780 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f84700013a0 tx=0x7f8470009040 comp rx=0 tx=0).stop 2024-01-16T15:00:56.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.301+0000 7f8486400640 1 -- 172.21.15.143:0/2820731136 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84801368c0 msgr2=0x7f84800cd1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:56.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.301+0000 7f8486400640 1 --2- 172.21.15.143:0/2820731136 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84801368c0 0x7f84800cd1d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f848006e680 tx=0x7f847800ec00 comp rx=0 tx=0).stop 2024-01-16T15:00:56.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.301+0000 7f8486400640 1 -- 172.21.15.143:0/2820731136 shutdown_connections 2024-01-16T15:00:56.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.301+0000 7f8486400640 1 --2- 172.21.15.143:0/2820731136 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f845806b2c0 0x7f845806d780 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:56.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.301+0000 7f8486400640 1 --2- 172.21.15.143:0/2820731136 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84801368c0 0x7f84800cd1d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:56.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.301+0000 7f8486400640 1 --2- 172.21.15.143:0/2820731136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8480143520 0x7f84800ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:56.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.301+0000 7f8486400640 1 --2- 172.21.15.143:0/2820731136 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84600047c0 0x7f84800cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:56.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.301+0000 7f8486400640 1 -- 172.21.15.143:0/2820731136 >> 172.21.15.143:0/2820731136 conn(0x7f84800cb830 msgr2=0x7f8480072220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:56.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.301+0000 7f8486400640 1 -- 172.21.15.143:0/2820731136 shutdown_connections 2024-01-16T15:00:56.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.301+0000 7f8486400640 1 -- 172.21.15.143:0/2820731136 wait complete. 2024-01-16T15:00:56.301 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-16T14:59:51.727720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-16T15:00:01.010472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-16T15:00:02.041228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-16T15:00:56.301 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-16T15:00:56.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.533+0000 7f15a7200640 1 Processor -- start 2024-01-16T15:00:56.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.533+0000 7f15a7200640 1 -- start start 2024-01-16T15:00:56.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.534+0000 7f15a7200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15a015a1c0 0x7f15a006d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:56.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.534+0000 7f15a7200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f15a00b0b40 0x7f15a00b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:56.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.534+0000 7f15a7200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f15a015ded0 0x7f15a015e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:56.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.534+0000 7f15a7200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a006d430 con 0x7f15a00b0b40 2024-01-16T15:00:56.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.534+0000 7f15a7200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a00b14e0 con 0x7f15a015ded0 2024-01-16T15:00:56.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.534+0000 7f15a7200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a015ec50 con 0x7f15a015a1c0 2024-01-16T15:00:56.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.534+0000 7f15a5400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f15a00b0b40 0x7f15a00b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:56.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.535+0000 7f15a5400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f15a00b0b40 0x7f15a00b0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47968/0 (socket says 172.21.15.143:47968) 2024-01-16T15:00:56.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.535+0000 7f15a5400640 1 -- 172.21.15.143:0/221366972 learned_addr learned my addr 172.21.15.143:0/221366972 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:56.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.535+0000 7f15a5e00640 1 --2- 172.21.15.143:0/221366972 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15a015a1c0 0x7f15a006d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:56.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.535+0000 7f15a6800640 1 --2- 172.21.15.143:0/221366972 >> v2:172.21.15.204:3300/0 conn(0x7f15a015ded0 0x7f15a015e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:56.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.535+0000 7f15a5400640 1 -- 172.21.15.143:0/221366972 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15a015a1c0 msgr2=0x7f15a006d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:56.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.535+0000 7f15a5400640 1 --2- 172.21.15.143:0/221366972 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15a015a1c0 0x7f15a006d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:56.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.535+0000 7f15a5400640 1 -- 172.21.15.143:0/221366972 >> v2:172.21.15.204:3300/0 conn(0x7f15a015ded0 msgr2=0x7f15a015e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:56.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.535+0000 7f15a5400640 1 --2- 172.21.15.143:0/221366972 >> v2:172.21.15.204:3300/0 conn(0x7f15a015ded0 0x7f15a015e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:56.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.535+0000 7f15a5400640 1 -- 172.21.15.143:0/221366972 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a006cb00 con 0x7f15a00b0b40 2024-01-16T15:00:56.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.536+0000 7f15a5400640 1 --2- 172.21.15.143:0/221366972 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15a00b0b40 0x7f15a00b0fa0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f15a00ccc20 tx=0x7f158802db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a81af24161c4450f server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:56.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.536+0000 7f15a4a00640 1 -- 172.21.15.143:0/221366972 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158802f8e0 con 0x7f15a00b0b40 2024-01-16T15:00:56.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.537+0000 7f15a4a00640 1 -- 172.21.15.143:0/221366972 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f158802fa40 con 0x7f15a00b0b40 2024-01-16T15:00:56.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.537+0000 7f15a4a00640 1 -- 172.21.15.143:0/221366972 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158802fdd0 con 0x7f15a00b0b40 2024-01-16T15:00:56.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.537+0000 7f15a7200640 1 -- 172.21.15.143:0/221366972 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15a00b0b40 msgr2=0x7f15a00b0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:56.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.537+0000 7f15a7200640 1 --2- 172.21.15.143:0/221366972 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15a00b0b40 0x7f15a00b0fa0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f15a00ccc20 tx=0x7f158802db20 comp rx=0 tx=0).stop 2024-01-16T15:00:56.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.537+0000 7f15a7200640 1 -- 172.21.15.143:0/221366972 shutdown_connections 2024-01-16T15:00:56.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.537+0000 7f15a7200640 1 --2- 172.21.15.143:0/221366972 >> v2:172.21.15.204:3300/0 conn(0x7f15a015ded0 0x7f15a015e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:56.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.537+0000 7f15a7200640 1 --2- 172.21.15.143:0/221366972 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15a00b0b40 0x7f15a00b0fa0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:56.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.537+0000 7f15a7200640 1 --2- 172.21.15.143:0/221366972 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15a015a1c0 0x7f15a006d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:56.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.537+0000 7f15a7200640 1 -- 172.21.15.143:0/221366972 >> 172.21.15.143:0/221366972 conn(0x7f15a00cbb40 msgr2=0x7f15a00cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:56.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.537+0000 7f15a7200640 1 -- 172.21.15.143:0/221366972 shutdown_connections 2024-01-16T15:00:56.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.537+0000 7f15a7200640 1 -- 172.21.15.143:0/221366972 wait complete. 2024-01-16T15:00:56.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.539+0000 7f15a7200640 1 Processor -- start 2024-01-16T15:00:56.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.539+0000 7f15a7200640 1 -- start start 2024-01-16T15:00:56.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.540+0000 7f15a7200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15a00b0b40 0x7f15a015d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:56.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.540+0000 7f15a7200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15a015a1c0 0x7f15a00c4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:56.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.540+0000 7f15a7200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15a015ded0 0x7f15a00c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:56.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.540+0000 7f15a7200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a015d6d0 con 0x7f15a015ded0 2024-01-16T15:00:56.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.540+0000 7f15a7200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a01f2e90 con 0x7f15a015a1c0 2024-01-16T15:00:56.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.540+0000 7f15a7200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a01f3170 con 0x7f15a00b0b40 2024-01-16T15:00:56.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.540+0000 7f15a6800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15a015ded0 0x7f15a00c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:56.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.540+0000 7f15a6800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15a015ded0 0x7f15a00c8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47980/0 (socket says 172.21.15.143:47980) 2024-01-16T15:00:56.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.540+0000 7f15a6800640 1 -- 172.21.15.143:0/3581502296 learned_addr learned my addr 172.21.15.143:0/3581502296 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:56.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.541+0000 7f15a5e00640 1 --2- 172.21.15.143:0/3581502296 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15a00b0b40 0x7f15a015d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:56.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.541+0000 7f15a5400640 1 --2- 172.21.15.143:0/3581502296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15a015a1c0 0x7f15a00c4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:56.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.541+0000 7f15a6800640 1 -- 172.21.15.143:0/3581502296 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15a00b0b40 msgr2=0x7f15a015d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:56.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.541+0000 7f15a6800640 1 --2- 172.21.15.143:0/3581502296 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15a00b0b40 0x7f15a015d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:56.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.541+0000 7f15a6800640 1 -- 172.21.15.143:0/3581502296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15a015a1c0 msgr2=0x7f15a00c4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:56.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.541+0000 7f15a6800640 1 --2- 172.21.15.143:0/3581502296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15a015a1c0 0x7f15a00c4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:56.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.541+0000 7f15a6800640 1 -- 172.21.15.143:0/3581502296 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15880094d0 con 0x7f15a015ded0 2024-01-16T15:00:56.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.541+0000 7f15a5e00640 1 --2- 172.21.15.143:0/3581502296 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15a00b0b40 0x7f15a015d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:56.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.541+0000 7f15a5400640 1 --2- 172.21.15.143:0/3581502296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15a015a1c0 0x7f15a00c4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:56.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.541+0000 7f15a6800640 1 --2- 172.21.15.143:0/3581502296 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15a015ded0 0x7f15a00c8360 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1598007e60 tx=0x7f159800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:56.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.541+0000 7f159e000640 1 -- 172.21.15.143:0/3581502296 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1598014420 con 0x7f15a015ded0 2024-01-16T15:00:56.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.541+0000 7f159e000640 1 -- 172.21.15.143:0/3581502296 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15980145a0 con 0x7f15a015ded0 2024-01-16T15:00:56.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.541+0000 7f159e000640 1 -- 172.21.15.143:0/3581502296 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1598014950 con 0x7f15a015ded0 2024-01-16T15:00:56.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.541+0000 7f15a7200640 1 -- 172.21.15.143:0/3581502296 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a01f3450 con 0x7f15a015ded0 2024-01-16T15:00:56.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.542+0000 7f15a7200640 1 -- 172.21.15.143:0/3581502296 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a00c5290 con 0x7f15a015ded0 2024-01-16T15:00:56.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.543+0000 7f15a7200640 1 -- 172.21.15.143:0/3581502296 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a01f3970 con 0x7f15a015ded0 2024-01-16T15:00:56.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.544+0000 7f159e000640 1 -- 172.21.15.143:0/3581502296 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f1598014ab0 con 0x7f15a015ded0 2024-01-16T15:00:56.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.545+0000 7f159e000640 1 --2- 172.21.15.143:0/3581502296 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f157806b2c0 0x7f157806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:56.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.545+0000 7f159e000640 1 -- 172.21.15.143:0/3581502296 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f159808a640 con 0x7f15a015ded0 2024-01-16T15:00:56.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.545+0000 7f15a5e00640 1 --2- 172.21.15.143:0/3581502296 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f157806b2c0 0x7f157806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:56.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.546+0000 7f15a5e00640 1 --2- 172.21.15.143:0/3581502296 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f157806b2c0 0x7f157806d780 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1590006fd0 tx=0x7f1590008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:56.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.549+0000 7f159e000640 1 -- 172.21.15.143:0/3581502296 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f15a01f3970 con 0x7f15a015ded0 2024-01-16T15:00:56.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.741+0000 7f15a7200640 1 -- 172.21.15.143:0/3581502296 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f15a01f3970 con 0x7f15a015ded0 2024-01-16T15:00:56.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.742+0000 7f159e000640 1 -- 172.21.15.143:0/3581502296 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f15a01f3970 con 0x7f15a015ded0 2024-01-16T15:00:56.723 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-16T15:00:56.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.745+0000 7f15a7200640 1 -- 172.21.15.143:0/3581502296 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f157806b2c0 msgr2=0x7f157806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:56.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.745+0000 7f15a7200640 1 --2- 172.21.15.143:0/3581502296 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f157806b2c0 0x7f157806d780 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1590006fd0 tx=0x7f1590008040 comp rx=0 tx=0).stop 2024-01-16T15:00:56.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.745+0000 7f15a7200640 1 -- 172.21.15.143:0/3581502296 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15a015ded0 msgr2=0x7f15a00c8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:56.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.745+0000 7f15a7200640 1 --2- 172.21.15.143:0/3581502296 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15a015ded0 0x7f15a00c8360 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1598007e60 tx=0x7f159800e4b0 comp rx=0 tx=0).stop 2024-01-16T15:00:56.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.745+0000 7f15a7200640 1 -- 172.21.15.143:0/3581502296 shutdown_connections 2024-01-16T15:00:56.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.745+0000 7f15a7200640 1 --2- 172.21.15.143:0/3581502296 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f157806b2c0 0x7f157806d780 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:56.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.745+0000 7f15a7200640 1 --2- 172.21.15.143:0/3581502296 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15a015ded0 0x7f15a00c8360 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:56.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.745+0000 7f15a7200640 1 --2- 172.21.15.143:0/3581502296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15a015a1c0 0x7f15a00c4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:56.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.745+0000 7f15a7200640 1 --2- 172.21.15.143:0/3581502296 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15a00b0b40 0x7f15a015d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:56.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.745+0000 7f15a7200640 1 -- 172.21.15.143:0/3581502296 >> 172.21.15.143:0/3581502296 conn(0x7f15a00cbb40 msgr2=0x7f15a00b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:56.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.746+0000 7f15a7200640 1 -- 172.21.15.143:0/3581502296 shutdown_connections 2024-01-16T15:00:56.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.746+0000 7f15a7200640 1 -- 172.21.15.143:0/3581502296 wait complete. 2024-01-16T15:00:56.744 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-16T15:00:56.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.974+0000 7fea28400640 1 Processor -- start 2024-01-16T15:00:56.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.974+0000 7fea28400640 1 -- start start 2024-01-16T15:00:56.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.976+0000 7fea28400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea2015b440 0x7fea200cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:56.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.976+0000 7fea28400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea200cc3a0 con 0x7fea200cbf20 2024-01-16T15:00:56.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.976+0000 7fea28400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea2015eea0 con 0x7fea200c59b0 2024-01-16T15:00:56.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.976+0000 7fea28400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea2015b820 con 0x7fea2015b440 2024-01-16T15:00:56.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.976+0000 7fea25000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fea200cbf20 0x7fea200c52a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43396/0 (socket says 172.21.15.143:43396) 2024-01-16T15:00:56.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.976+0000 7fea25000640 1 -- 172.21.15.143:0/1833554595 learned_addr learned my addr 172.21.15.143:0/1833554595 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:56.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.977+0000 7fea25a00640 1 --2- 172.21.15.143:0/1833554595 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea2015b440 0x7fea200cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:56.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.977+0000 7fea1fe00640 1 -- 172.21.15.143:0/1833554595 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 19571792 0 0) 0x7fea08001110 con 0x7fea200cbf20 2024-01-16T15:00:56.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.977+0000 7fea1fe00640 1 -- 172.21.15.143:0/1833554595 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea00003520 con 0x7fea200cbf20 2024-01-16T15:00:56.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.977+0000 7fea1fe00640 1 -- 172.21.15.143:0/1833554595 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 178847752 0 0) 0x7fea00003520 con 0x7fea200cbf20 2024-01-16T15:00:56.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.977+0000 7fea1fe00640 1 -- 172.21.15.143:0/1833554595 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea2015b440 msgr2=0x7fea200cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:56.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.977+0000 7fea1fe00640 1 --2- 172.21.15.143:0/1833554595 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea2015b440 0x7fea200cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:56.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.977+0000 7fea1fe00640 1 -- 172.21.15.143:0/1833554595 >> v1:172.21.15.204:6789/0 conn(0x7fea200c59b0 legacy=0x7fea2006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:56.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.977+0000 7fea1fe00640 1 -- 172.21.15.143:0/1833554595 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea200abf30 con 0x7fea200cbf20 2024-01-16T15:00:56.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.977+0000 7fea1fe00640 1 -- 172.21.15.143:0/1833554595 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7fea08001800 con 0x7fea200cbf20 2024-01-16T15:00:56.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.977+0000 7fea1fe00640 1 -- 172.21.15.143:0/1833554595 >> v1:172.21.15.143:6789/0 conn(0x7fea200cbf20 legacy=0x7fea200c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:56.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.977+0000 7fea1fe00640 1 --2- 172.21.15.143:0/1833554595 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea000039a0 0x7fea00003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:56.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.977+0000 7fea1fe00640 1 --2- 172.21.15.143:0/1833554595 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea2015b440 0x7fea000047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:56.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.978+0000 7fea1fe00640 1 --2- 172.21.15.143:0/1833554595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea00004d00 0x7fea00005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:56.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.978+0000 7fea1fe00640 1 -- 172.21.15.143:0/1833554595 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2015b820 con 0x7fea000039a0 2024-01-16T15:00:56.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.978+0000 7fea1fe00640 1 -- 172.21.15.143:0/1833554595 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0000bab0 con 0x7fea00004d00 2024-01-16T15:00:56.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.978+0000 7fea1fe00640 1 -- 172.21.15.143:0/1833554595 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea0000bc70 con 0x7fea2015b440 2024-01-16T15:00:56.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.978+0000 7fea25a00640 1 --2- 172.21.15.143:0/1833554595 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea000039a0 0x7fea00003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:56.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.978+0000 7fea25a00640 1 -- 172.21.15.143:0/1833554595 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea2015b440 msgr2=0x7fea000047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:56.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.978+0000 7fea25a00640 1 --2- 172.21.15.143:0/1833554595 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea2015b440 0x7fea000047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:56.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.978+0000 7fea25a00640 1 -- 172.21.15.143:0/1833554595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea00004d00 msgr2=0x7fea00005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:56.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.978+0000 7fea25a00640 1 --2- 172.21.15.143:0/1833554595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea00004d00 0x7fea00005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:56.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.978+0000 7fea25a00640 1 -- 172.21.15.143:0/1833554595 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea00003520 con 0x7fea000039a0 2024-01-16T15:00:56.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.979+0000 7fea25a00640 1 --2- 172.21.15.143:0/1833554595 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea000039a0 0x7fea00003dc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fea10009ac0 tx=0x7fea000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67e806f6c4a3d994 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:56.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.979+0000 7fea1fe00640 1 -- 172.21.15.143:0/1833554595 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1002c630 con 0x7fea000039a0 2024-01-16T15:00:56.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.979+0000 7fea1fe00640 1 -- 172.21.15.143:0/1833554595 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1002c790 con 0x7fea000039a0 2024-01-16T15:00:56.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.979+0000 7fea1fe00640 1 -- 172.21.15.143:0/1833554595 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1002cb20 con 0x7fea000039a0 2024-01-16T15:00:56.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.980+0000 7fea28400640 1 -- 172.21.15.143:0/1833554595 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea000039a0 msgr2=0x7fea00003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:56.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.980+0000 7fea28400640 1 --2- 172.21.15.143:0/1833554595 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea000039a0 0x7fea00003dc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fea10009ac0 tx=0x7fea000057f0 comp rx=0 tx=0).stop 2024-01-16T15:00:56.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.980+0000 7fea26400640 1 -- 172.21.15.143:0/1833554595 reap_dead start 2024-01-16T15:00:56.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.980+0000 7fea28400640 1 -- 172.21.15.143:0/1833554595 shutdown_connections 2024-01-16T15:00:56.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.980+0000 7fea28400640 1 -- 172.21.15.143:0/1833554595 >> 172.21.15.143:0/1833554595 conn(0x7fea200b0c50 msgr2=0x7fea200b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:56.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.980+0000 7fea28400640 1 -- 172.21.15.143:0/1833554595 shutdown_connections 2024-01-16T15:00:56.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.980+0000 7fea28400640 1 -- 172.21.15.143:0/1833554595 wait complete. 2024-01-16T15:00:56.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.982+0000 7fea28400640 1 Processor -- start 2024-01-16T15:00:56.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.982+0000 7fea28400640 1 -- start start 2024-01-16T15:00:56.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.983+0000 7fea28400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea000039a0 0x7fea200ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:56.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.983+0000 7fea28400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea200ce790 0x7fea20156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:56.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.983+0000 7fea28400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea201544f0 0x7fea201f2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:56.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.983+0000 7fea28400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea201565a0 con 0x7fea201544f0 2024-01-16T15:00:56.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.983+0000 7fea28400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea200cd6f0 con 0x7fea200ce790 2024-01-16T15:00:56.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.983+0000 7fea28400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea200cd9d0 con 0x7fea000039a0 2024-01-16T15:00:56.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.983+0000 7fea26400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea201544f0 0x7fea201f2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:56.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.983+0000 7fea26400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea201544f0 0x7fea201f2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48002/0 (socket says 172.21.15.143:48002) 2024-01-16T15:00:56.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.983+0000 7fea26400640 1 -- 172.21.15.143:0/2636941150 learned_addr learned my addr 172.21.15.143:0/2636941150 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:56.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.984+0000 7fea25a00640 1 --2- 172.21.15.143:0/2636941150 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea000039a0 0x7fea200ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:56.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.984+0000 7fea25000640 1 --2- 172.21.15.143:0/2636941150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea200ce790 0x7fea20156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:56.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.984+0000 7fea26400640 1 -- 172.21.15.143:0/2636941150 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea000039a0 msgr2=0x7fea200ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:56.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.984+0000 7fea26400640 1 --2- 172.21.15.143:0/2636941150 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea000039a0 0x7fea200ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:56.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.984+0000 7fea26400640 1 -- 172.21.15.143:0/2636941150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea200ce790 msgr2=0x7fea20156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:56.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.984+0000 7fea26400640 1 --2- 172.21.15.143:0/2636941150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea200ce790 0x7fea20156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:56.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.984+0000 7fea26400640 1 -- 172.21.15.143:0/2636941150 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea200cdcb0 con 0x7fea201544f0 2024-01-16T15:00:56.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.984+0000 7fea26400640 1 --2- 172.21.15.143:0/2636941150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea201544f0 0x7fea201f2030 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fea200ccb40 tx=0x7fea1800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:56.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.984+0000 7fea1d600640 1 -- 172.21.15.143:0/2636941150 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea18005d40 con 0x7fea201544f0 2024-01-16T15:00:56.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.984+0000 7fea1d600640 1 -- 172.21.15.143:0/2636941150 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1800ced0 con 0x7fea201544f0 2024-01-16T15:00:56.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.984+0000 7fea1d600640 1 -- 172.21.15.143:0/2636941150 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1800a850 con 0x7fea201544f0 2024-01-16T15:00:56.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.984+0000 7fea28400640 1 -- 172.21.15.143:0/2636941150 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea200cdf90 con 0x7fea201544f0 2024-01-16T15:00:56.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.985+0000 7fea28400640 1 -- 172.21.15.143:0/2636941150 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea20154970 con 0x7fea201544f0 2024-01-16T15:00:56.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.986+0000 7fea28400640 1 -- 172.21.15.143:0/2636941150 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea20154b90 con 0x7fea201544f0 2024-01-16T15:00:56.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.987+0000 7fea1d600640 1 -- 172.21.15.143:0/2636941150 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fea1800a9b0 con 0x7fea201544f0 2024-01-16T15:00:56.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.987+0000 7fea1d600640 1 --2- 172.21.15.143:0/2636941150 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe9f806b2f0 0x7fe9f806d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:56.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.988+0000 7fea1d600640 1 -- 172.21.15.143:0/2636941150 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fea18088d70 con 0x7fea201544f0 2024-01-16T15:00:56.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.988+0000 7fea25a00640 1 --2- 172.21.15.143:0/2636941150 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe9f806b2f0 0x7fe9f806d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:56.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.989+0000 7fea25a00640 1 --2- 172.21.15.143:0/2636941150 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe9f806b2f0 0x7fe9f806d7b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fea10036000 tx=0x7fea10030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:56.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:56.992+0000 7fea1d600640 1 -- 172.21.15.143:0/2636941150 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fea18053b60 con 0x7fea201544f0 2024-01-16T15:00:57.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.180+0000 7fea28400640 1 -- 172.21.15.143:0/2636941150 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fea2004e8b0 con 0x7fea201544f0 2024-01-16T15:00:57.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.181+0000 7fea1d600640 1 -- 172.21.15.143:0/2636941150 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fea18057180 con 0x7fea201544f0 2024-01-16T15:00:57.162 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2024-01-16T15:00:57.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.183+0000 7fea28400640 1 -- 172.21.15.143:0/2636941150 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe9f806b2f0 msgr2=0x7fe9f806d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:57.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.184+0000 7fea28400640 1 --2- 172.21.15.143:0/2636941150 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe9f806b2f0 0x7fe9f806d7b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fea10036000 tx=0x7fea10030000 comp rx=0 tx=0).stop 2024-01-16T15:00:57.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.184+0000 7fea28400640 1 -- 172.21.15.143:0/2636941150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea201544f0 msgr2=0x7fea201f2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:57.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.184+0000 7fea28400640 1 --2- 172.21.15.143:0/2636941150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea201544f0 0x7fea201f2030 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fea200ccb40 tx=0x7fea1800cb90 comp rx=0 tx=0).stop 2024-01-16T15:00:57.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.184+0000 7fea28400640 1 -- 172.21.15.143:0/2636941150 shutdown_connections 2024-01-16T15:00:57.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.184+0000 7fea28400640 1 --2- 172.21.15.143:0/2636941150 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe9f806b2f0 0x7fe9f806d7b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:57.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.184+0000 7fea28400640 1 --2- 172.21.15.143:0/2636941150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea201544f0 0x7fea201f2030 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:57.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.184+0000 7fea28400640 1 --2- 172.21.15.143:0/2636941150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea200ce790 0x7fea20156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:57.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.184+0000 7fea28400640 1 --2- 172.21.15.143:0/2636941150 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea000039a0 0x7fea200ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:57.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.184+0000 7fea28400640 1 -- 172.21.15.143:0/2636941150 >> 172.21.15.143:0/2636941150 conn(0x7fea200b0c50 msgr2=0x7fea20071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:57.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.184+0000 7fea28400640 1 -- 172.21.15.143:0/2636941150 shutdown_connections 2024-01-16T15:00:57.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.184+0000 7fea28400640 1 -- 172.21.15.143:0/2636941150 wait complete. 2024-01-16T15:00:57.183 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-16T15:00:57.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.416+0000 7f1232800640 1 Processor -- start 2024-01-16T15:00:57.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.416+0000 7f1232800640 1 -- start start 2024-01-16T15:00:57.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.418+0000 7f1232800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f122c06de10 0x7f122c15e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:57.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.418+0000 7f1232800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f122c0c49f0 0x7f122c0705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:57.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.418+0000 7f1232800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f122c15ec90 con 0x7f122c072160 2024-01-16T15:00:57.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.418+0000 7f1232800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f122c0cd530 con 0x7f122c06de10 2024-01-16T15:00:57.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.418+0000 7f1232800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f122c0c4ed0 con 0x7f122c0c49f0 2024-01-16T15:00:57.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.418+0000 7f122b400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f122c072160 0x7f122c06d700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43404/0 (socket says 172.21.15.143:43404) 2024-01-16T15:00:57.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.418+0000 7f122b400640 1 -- 172.21.15.143:0/4240529846 learned_addr learned my addr 172.21.15.143:0/4240529846 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:57.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.418+0000 7f1223e00640 1 --2- 172.21.15.143:0/4240529846 >> v2:172.21.15.143:3300/0 conn(0x7f122c06de10 0x7f122c15e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:57.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.418+0000 7f122be00640 1 --2- 172.21.15.143:0/4240529846 >> v2:172.21.15.204:3300/0 conn(0x7f122c0c49f0 0x7f122c0705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:57.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.419+0000 7f1223e00640 1 -- 172.21.15.143:0/4240529846 >> v2:172.21.15.204:3300/0 conn(0x7f122c0c49f0 msgr2=0x7f122c0705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:57.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.419+0000 7f1223e00640 1 --2- 172.21.15.143:0/4240529846 >> v2:172.21.15.204:3300/0 conn(0x7f122c0c49f0 0x7f122c0705c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:57.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.419+0000 7f1223e00640 1 -- 172.21.15.143:0/4240529846 >> v1:172.21.15.143:6789/0 conn(0x7f122c072160 legacy=0x7f122c06d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:57.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.419+0000 7f122aa00640 1 -- 172.21.15.143:0/4240529846 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1525890552 0 0) 0x7f121c001110 con 0x7f122c072160 2024-01-16T15:00:57.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.419+0000 7f1223e00640 1 -- 172.21.15.143:0/4240529846 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f122c0c51b0 con 0x7f122c06de10 2024-01-16T15:00:57.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.420+0000 7f1223e00640 1 --2- 172.21.15.143:0/4240529846 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f122c06de10 0x7f122c15e750 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f122c072a30 tx=0x7f121402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d54090cfaeeb1ae server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:57.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.420+0000 7f122aa00640 1 -- 172.21.15.143:0/4240529846 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1214031830 con 0x7f122c06de10 2024-01-16T15:00:57.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.420+0000 7f122aa00640 1 -- 172.21.15.143:0/4240529846 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1214031990 con 0x7f122c06de10 2024-01-16T15:00:57.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.420+0000 7f122aa00640 1 -- 172.21.15.143:0/4240529846 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1214031d20 con 0x7f122c06de10 2024-01-16T15:00:57.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.420+0000 7f1232800640 1 -- 172.21.15.143:0/4240529846 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f122c06de10 msgr2=0x7f122c15e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:57.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.421+0000 7f1232800640 1 --2- 172.21.15.143:0/4240529846 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f122c06de10 0x7f122c15e750 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f122c072a30 tx=0x7f121402b8d0 comp rx=0 tx=0).stop 2024-01-16T15:00:57.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.421+0000 7f1232800640 1 -- 172.21.15.143:0/4240529846 shutdown_connections 2024-01-16T15:00:57.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.421+0000 7f1232800640 1 --2- 172.21.15.143:0/4240529846 >> v2:172.21.15.204:3300/0 conn(0x7f122c0c49f0 0x7f122c0705c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:57.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.421+0000 7f1232800640 1 --2- 172.21.15.143:0/4240529846 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f122c06de10 0x7f122c15e750 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:57.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.421+0000 7f1232800640 1 -- 172.21.15.143:0/4240529846 >> 172.21.15.143:0/4240529846 conn(0x7f122c0b0c50 msgr2=0x7f122c0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:57.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.421+0000 7f1232800640 1 -- 172.21.15.143:0/4240529846 shutdown_connections 2024-01-16T15:00:57.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.421+0000 7f1232800640 1 -- 172.21.15.143:0/4240529846 wait complete. 2024-01-16T15:00:57.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.422+0000 7f1232800640 1 Processor -- start 2024-01-16T15:00:57.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.423+0000 7f1232800640 1 -- start start 2024-01-16T15:00:57.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.423+0000 7f1232800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f122c06de10 0x7f122c156c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:57.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.423+0000 7f1232800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f122c072160 0x7f122c165740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:57.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.423+0000 7f1232800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f122c0c49f0 0x7f122c1661e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:57.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.423+0000 7f1232800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122c163d10 con 0x7f122c0c49f0 2024-01-16T15:00:57.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.423+0000 7f1232800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122c163ff0 con 0x7f122c072160 2024-01-16T15:00:57.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.424+0000 7f1232800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f122c166f60 con 0x7f122c06de10 2024-01-16T15:00:57.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.424+0000 7f122be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f122c0c49f0 0x7f122c1661e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:57.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.424+0000 7f122be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f122c0c49f0 0x7f122c1661e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48026/0 (socket says 172.21.15.143:48026) 2024-01-16T15:00:57.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.424+0000 7f122be00640 1 -- 172.21.15.143:0/2690423540 learned_addr learned my addr 172.21.15.143:0/2690423540 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:57.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.424+0000 7f122be00640 1 -- 172.21.15.143:0/2690423540 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f122c06de10 msgr2=0x7f122c156c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:57.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.424+0000 7f122b400640 1 --2- 172.21.15.143:0/2690423540 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f122c06de10 0x7f122c156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:57.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.424+0000 7f122be00640 1 --2- 172.21.15.143:0/2690423540 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f122c06de10 0x7f122c156c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:57.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.424+0000 7f1223e00640 1 --2- 172.21.15.143:0/2690423540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f122c072160 0x7f122c165740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:57.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.424+0000 7f122be00640 1 -- 172.21.15.143:0/2690423540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f122c072160 msgr2=0x7f122c165740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:57.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.424+0000 7f122be00640 1 --2- 172.21.15.143:0/2690423540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f122c072160 0x7f122c165740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:57.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.424+0000 7f122be00640 1 -- 172.21.15.143:0/2690423540 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1214009400 con 0x7f122c0c49f0 2024-01-16T15:00:57.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.425+0000 7f122be00640 1 --2- 172.21.15.143:0/2690423540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f122c0c49f0 0x7f122c1661e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1224009fd0 tx=0x7f122400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:57.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.425+0000 7f1223400640 1 -- 172.21.15.143:0/2690423540 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1224014420 con 0x7f122c0c49f0 2024-01-16T15:00:57.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.425+0000 7f1223400640 1 -- 172.21.15.143:0/2690423540 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1224014580 con 0x7f122c0c49f0 2024-01-16T15:00:57.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.425+0000 7f1223400640 1 -- 172.21.15.143:0/2690423540 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1224014930 con 0x7f122c0c49f0 2024-01-16T15:00:57.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.425+0000 7f1232800640 1 -- 172.21.15.143:0/2690423540 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f122c167240 con 0x7f122c0c49f0 2024-01-16T15:00:57.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.425+0000 7f1232800640 1 -- 172.21.15.143:0/2690423540 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f122c1571b0 con 0x7f122c0c49f0 2024-01-16T15:00:57.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.426+0000 7f1232800640 1 -- 172.21.15.143:0/2690423540 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f122c167430 con 0x7f122c0c49f0 2024-01-16T15:00:57.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.427+0000 7f1223400640 1 -- 172.21.15.143:0/2690423540 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f1224014a90 con 0x7f122c0c49f0 2024-01-16T15:00:57.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.428+0000 7f1223400640 1 --2- 172.21.15.143:0/2690423540 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f120406b2f0 0x7f120406d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:57.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.428+0000 7f1223400640 1 -- 172.21.15.143:0/2690423540 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f122408a4a0 con 0x7f122c0c49f0 2024-01-16T15:00:57.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.428+0000 7f122b400640 1 --2- 172.21.15.143:0/2690423540 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f120406b2f0 0x7f120406d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:57.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.429+0000 7f122b400640 1 --2- 172.21.15.143:0/2690423540 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f120406b2f0 0x7f120406d7b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f121c000f60 tx=0x7f121c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:57.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.432+0000 7f1223400640 1 -- 172.21.15.143:0/2690423540 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1224055320 con 0x7f122c0c49f0 2024-01-16T15:00:57.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.620+0000 7f1232800640 1 -- 172.21.15.143:0/2690423540 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f122c04e8b0 con 0x7f122c0c49f0 2024-01-16T15:00:57.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.621+0000 7f1223400640 1 -- 172.21.15.143:0/2690423540 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1224058940 con 0x7f122c0c49f0 2024-01-16T15:00:57.602 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2024-01-16T15:00:57.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.623+0000 7f1232800640 1 -- 172.21.15.143:0/2690423540 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f120406b2f0 msgr2=0x7f120406d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:57.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.624+0000 7f1232800640 1 --2- 172.21.15.143:0/2690423540 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f120406b2f0 0x7f120406d7b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f121c000f60 tx=0x7f121c009040 comp rx=0 tx=0).stop 2024-01-16T15:00:57.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.624+0000 7f1232800640 1 -- 172.21.15.143:0/2690423540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f122c0c49f0 msgr2=0x7f122c1661e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:57.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.624+0000 7f1232800640 1 --2- 172.21.15.143:0/2690423540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f122c0c49f0 0x7f122c1661e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1224009fd0 tx=0x7f122400e4b0 comp rx=0 tx=0).stop 2024-01-16T15:00:57.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.624+0000 7f1232800640 1 -- 172.21.15.143:0/2690423540 shutdown_connections 2024-01-16T15:00:57.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.624+0000 7f1232800640 1 --2- 172.21.15.143:0/2690423540 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f120406b2f0 0x7f120406d7b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:57.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.624+0000 7f1232800640 1 --2- 172.21.15.143:0/2690423540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f122c0c49f0 0x7f122c1661e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:57.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.624+0000 7f1232800640 1 --2- 172.21.15.143:0/2690423540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f122c072160 0x7f122c165740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:57.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.624+0000 7f1232800640 1 --2- 172.21.15.143:0/2690423540 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f122c06de10 0x7f122c156c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:57.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.624+0000 7f1232800640 1 -- 172.21.15.143:0/2690423540 >> 172.21.15.143:0/2690423540 conn(0x7f122c0b0c50 msgr2=0x7f122c0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:57.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.624+0000 7f1232800640 1 -- 172.21.15.143:0/2690423540 shutdown_connections 2024-01-16T15:00:57.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.624+0000 7f1232800640 1 -- 172.21.15.143:0/2690423540 wait complete. 2024-01-16T15:00:57.624 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-16T15:00:57.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.856+0000 7fe8d8a00640 1 Processor -- start 2024-01-16T15:00:57.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.857+0000 7fe8d8a00640 1 -- start start 2024-01-16T15:00:57.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.858+0000 7fe8d8a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe8d413d120 0x7fe8d415af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:57.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.858+0000 7fe8d8a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8d40cec00 con 0x7fe8d406d6c0 2024-01-16T15:00:57.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.858+0000 7fe8d8a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8d40b1890 con 0x7fe8d406db20 2024-01-16T15:00:57.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.858+0000 7fe8d8a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8d413f9e0 con 0x7fe8d413d120 2024-01-16T15:00:57.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.858+0000 7fe8c9c00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe8d406d6c0 0x7fe8d40b0a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43406/0 (socket says 172.21.15.143:43406) 2024-01-16T15:00:57.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.858+0000 7fe8c9c00640 1 -- 172.21.15.143:0/1214248706 learned_addr learned my addr 172.21.15.143:0/1214248706 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:57.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.859+0000 7fe8d2800640 1 --2- 172.21.15.143:0/1214248706 >> v2:172.21.15.204:3300/0 conn(0x7fe8d413d120 0x7fe8d415af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:57.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.859+0000 7fe8d1e00640 1 -- 172.21.15.143:0/1214248706 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3779393741 0 0) 0x7fe8d40cec00 con 0x7fe8d406d6c0 2024-01-16T15:00:57.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.859+0000 7fe8d1e00640 1 -- 172.21.15.143:0/1214248706 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8b4005730 con 0x7fe8d406d6c0 2024-01-16T15:00:57.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.859+0000 7fe8d1e00640 1 -- 172.21.15.143:0/1214248706 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1872265429 0 0) 0x7fe8b4005730 con 0x7fe8d406d6c0 2024-01-16T15:00:57.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.859+0000 7fe8d1e00640 1 -- 172.21.15.143:0/1214248706 >> v2:172.21.15.204:3300/0 conn(0x7fe8d413d120 msgr2=0x7fe8d415af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:57.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.859+0000 7fe8d1e00640 1 --2- 172.21.15.143:0/1214248706 >> v2:172.21.15.204:3300/0 conn(0x7fe8d413d120 0x7fe8d415af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:57.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.859+0000 7fe8d1e00640 1 -- 172.21.15.143:0/1214248706 >> v1:172.21.15.204:6789/0 conn(0x7fe8d406db20 legacy=0x7fe8d40b1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:57.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.859+0000 7fe8d1e00640 1 -- 172.21.15.143:0/1214248706 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d413fcc0 con 0x7fe8d406d6c0 2024-01-16T15:00:57.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.859+0000 7fe8d1e00640 1 -- 172.21.15.143:0/1214248706 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7fe8bc001670 con 0x7fe8d406d6c0 2024-01-16T15:00:57.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.859+0000 7fe8d1e00640 1 -- 172.21.15.143:0/1214248706 >> v1:172.21.15.143:6789/0 conn(0x7fe8d406d6c0 legacy=0x7fe8d40b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:57.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.859+0000 7fe8d1e00640 1 --2- 172.21.15.143:0/1214248706 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8b4005b80 0x7fe8b4005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:57.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.859+0000 7fe8d1e00640 1 --2- 172.21.15.143:0/1214248706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8b40069d0 0x7fe8b4006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:57.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.860+0000 7fe8d1e00640 1 --2- 172.21.15.143:0/1214248706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8b4007390 0x7fe8b400db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:57.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.860+0000 7fe8d1e00640 1 -- 172.21.15.143:0/1214248706 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d413f9e0 con 0x7fe8b4007390 2024-01-16T15:00:57.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.860+0000 7fe8d1e00640 1 -- 172.21.15.143:0/1214248706 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b400e330 con 0x7fe8b40069d0 2024-01-16T15:00:57.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.860+0000 7fe8d1e00640 1 -- 172.21.15.143:0/1214248706 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b400e4f0 con 0x7fe8b4005b80 2024-01-16T15:00:57.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.860+0000 7fe8d3200640 1 --2- 172.21.15.143:0/1214248706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8b4007390 0x7fe8b400db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:57.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.860+0000 7fe8d2800640 1 --2- 172.21.15.143:0/1214248706 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8b4005b80 0x7fe8b4005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:57.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.860+0000 7fe8d3200640 1 -- 172.21.15.143:0/1214248706 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8b4005b80 msgr2=0x7fe8b4005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:57.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.860+0000 7fe8d3200640 1 --2- 172.21.15.143:0/1214248706 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8b4005b80 0x7fe8b4005fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:57.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.860+0000 7fe8d3200640 1 -- 172.21.15.143:0/1214248706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8b40069d0 msgr2=0x7fe8b4006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:57.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.860+0000 7fe8d3200640 1 --2- 172.21.15.143:0/1214248706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8b40069d0 0x7fe8b4006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:57.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.860+0000 7fe8d3200640 1 -- 172.21.15.143:0/1214248706 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b4005730 con 0x7fe8b4007390 2024-01-16T15:00:57.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.861+0000 7fe8d3200640 1 -- 172.21.15.143:0/1214248706 reap_dead start 2024-01-16T15:00:57.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.861+0000 7fe8d3200640 1 --2- 172.21.15.143:0/1214248706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8b4007390 0x7fe8b400db90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc001e50 tx=0x7fe8b4007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca476eb80651892a server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:57.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.862+0000 7fe8d1e00640 1 -- 172.21.15.143:0/1214248706 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc003b70 con 0x7fe8b4007390 2024-01-16T15:00:57.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.862+0000 7fe8d1e00640 1 -- 172.21.15.143:0/1214248706 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8cc003d10 con 0x7fe8b4007390 2024-01-16T15:00:57.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.862+0000 7fe8d1e00640 1 -- 172.21.15.143:0/1214248706 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc005310 con 0x7fe8b4007390 2024-01-16T15:00:57.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.862+0000 7fe8d8a00640 1 -- 172.21.15.143:0/1214248706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8b4007390 msgr2=0x7fe8b400db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:57.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.862+0000 7fe8d8a00640 1 --2- 172.21.15.143:0/1214248706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8b4007390 0x7fe8b400db90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc001e50 tx=0x7fe8b4007a00 comp rx=0 tx=0).stop 2024-01-16T15:00:57.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.862+0000 7fe8d8a00640 1 -- 172.21.15.143:0/1214248706 shutdown_connections 2024-01-16T15:00:57.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.862+0000 7fe8d8a00640 1 --2- 172.21.15.143:0/1214248706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8b4007390 0x7fe8b400db90 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:57.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.862+0000 7fe8d8a00640 1 -- 172.21.15.143:0/1214248706 >> 172.21.15.143:0/1214248706 conn(0x7fe8d40cb830 msgr2=0x7fe8d40cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:57.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.862+0000 7fe8d8a00640 1 -- 172.21.15.143:0/1214248706 shutdown_connections 2024-01-16T15:00:57.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.862+0000 7fe8d8a00640 1 -- 172.21.15.143:0/1214248706 wait complete. 2024-01-16T15:00:57.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.864+0000 7fe8d8a00640 1 Processor -- start 2024-01-16T15:00:57.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.864+0000 7fe8d8a00640 1 -- start start 2024-01-16T15:00:57.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.865+0000 7fe8d8a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8b4007390 0x7fe8d413c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:57.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.865+0000 7fe8d8a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8d4142b80 0x7fe8d4142fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:57.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.865+0000 7fe8d8a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8d415bf00 0x7fe8d415c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:57.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.865+0000 7fe8d8a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d415c8f0 con 0x7fe8b4007390 2024-01-16T15:00:57.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.865+0000 7fe8d8a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d4143660 con 0x7fe8d415bf00 2024-01-16T15:00:57.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.865+0000 7fe8d8a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d40aea50 con 0x7fe8d4142b80 2024-01-16T15:00:57.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.866+0000 7fe8d2800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8b4007390 0x7fe8d413c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:57.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.866+0000 7fe8d2800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8b4007390 0x7fe8d413c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48056/0 (socket says 172.21.15.143:48056) 2024-01-16T15:00:57.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.866+0000 7fe8d2800640 1 -- 172.21.15.143:0/404586449 learned_addr learned my addr 172.21.15.143:0/404586449 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:57.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.866+0000 7fe8d2800640 1 -- 172.21.15.143:0/404586449 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8d4142b80 msgr2=0x7fe8d4142fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:57.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.866+0000 7fe8d2800640 1 --2- 172.21.15.143:0/404586449 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8d4142b80 0x7fe8d4142fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:57.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.866+0000 7fe8d2800640 1 -- 172.21.15.143:0/404586449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8d415bf00 msgr2=0x7fe8d415c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:57.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.866+0000 7fe8d2800640 1 --2- 172.21.15.143:0/404586449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8d415bf00 0x7fe8d415c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:57.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.866+0000 7fe8d2800640 1 -- 172.21.15.143:0/404586449 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d41408d0 con 0x7fe8b4007390 2024-01-16T15:00:57.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.866+0000 7fe8d2800640 1 --2- 172.21.15.143:0/404586449 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8b4007390 0x7fe8d413c660 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4010860 tx=0x7fe8c4010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:57.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.867+0000 7fe8cb400640 1 -- 172.21.15.143:0/404586449 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4009610 con 0x7fe8b4007390 2024-01-16T15:00:57.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.867+0000 7fe8cb400640 1 -- 172.21.15.143:0/404586449 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c400eb60 con 0x7fe8b4007390 2024-01-16T15:00:57.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.867+0000 7fe8cb400640 1 -- 172.21.15.143:0/404586449 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c401c400 con 0x7fe8b4007390 2024-01-16T15:00:57.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.867+0000 7fe8d8a00640 1 -- 172.21.15.143:0/404586449 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d4140bb0 con 0x7fe8b4007390 2024-01-16T15:00:57.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.867+0000 7fe8d8a00640 1 -- 172.21.15.143:0/404586449 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d4141170 con 0x7fe8b4007390 2024-01-16T15:00:57.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.868+0000 7fe8d8a00640 1 -- 172.21.15.143:0/404586449 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d415e8b0 con 0x7fe8b4007390 2024-01-16T15:00:57.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.869+0000 7fe8cb400640 1 -- 172.21.15.143:0/404586449 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fe8c401c560 con 0x7fe8b4007390 2024-01-16T15:00:57.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.870+0000 7fe8cb400640 1 --2- 172.21.15.143:0/404586449 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe8ac06b2f0 0x7fe8ac06d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:57.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.870+0000 7fe8cb400640 1 -- 172.21.15.143:0/404586449 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fe8c408a190 con 0x7fe8b4007390 2024-01-16T15:00:57.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.870+0000 7fe8c9c00640 1 --2- 172.21.15.143:0/404586449 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe8ac06b2f0 0x7fe8ac06d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:57.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.871+0000 7fe8c9c00640 1 --2- 172.21.15.143:0/404586449 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe8ac06b2f0 0x7fe8ac06d7b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0011e0 tx=0x7fe8bc0014d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:57.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:57.874+0000 7fe8cb400640 1 -- 172.21.15.143:0/404586449 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe8c4055020 con 0x7fe8b4007390 2024-01-16T15:00:58.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.101+0000 7fe8d8a00640 1 -- 172.21.15.143:0/404586449 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8d404e840 con 0x7fe8b4007390 2024-01-16T15:00:58.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.102+0000 7fe8cb400640 1 -- 172.21.15.143:0/404586449 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4916 (secure 0 0 0) 0x7fe8c4058640 con 0x7fe8b4007390 2024-01-16T15:00:58.083 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:00:58.083 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3599169408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3599169408},{"type":"v1","addr":"172.21.15.204:6834","nonce":3599169408}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/859082411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":859082411},{"type":"v1","addr":"172.21.15.204:6835","nonce":859082411}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6836/1564738611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1564738611},{"type":"v1","addr":"172.21.15.143:6836","nonce":1564738611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-16T15:00:03.050082+0000","modified":"2024-01-16T15:00:06.109664+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/3120706283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6835","nonce":3120706283},{"type":"v1","addr":"172.21.15.143:6837","nonce":3120706283}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-16T15:00:58.083 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 8 2024-01-16T15:00:58.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.105+0000 7fe8d8a00640 1 -- 172.21.15.143:0/404586449 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe8ac06b2f0 msgr2=0x7fe8ac06d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:58.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.105+0000 7fe8d8a00640 1 --2- 172.21.15.143:0/404586449 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe8ac06b2f0 0x7fe8ac06d7b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0011e0 tx=0x7fe8bc0014d0 comp rx=0 tx=0).stop 2024-01-16T15:00:58.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.105+0000 7fe8d8a00640 1 -- 172.21.15.143:0/404586449 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8b4007390 msgr2=0x7fe8d413c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:58.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.105+0000 7fe8d8a00640 1 --2- 172.21.15.143:0/404586449 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8b4007390 0x7fe8d413c660 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4010860 tx=0x7fe8c4010d20 comp rx=0 tx=0).stop 2024-01-16T15:00:58.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.105+0000 7fe8d8a00640 1 -- 172.21.15.143:0/404586449 shutdown_connections 2024-01-16T15:00:58.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.105+0000 7fe8d8a00640 1 --2- 172.21.15.143:0/404586449 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe8ac06b2f0 0x7fe8ac06d7b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:58.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.105+0000 7fe8d8a00640 1 --2- 172.21.15.143:0/404586449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8d415bf00 0x7fe8d415c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:58.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.105+0000 7fe8d8a00640 1 --2- 172.21.15.143:0/404586449 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8d4142b80 0x7fe8d4142fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:58.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.105+0000 7fe8d8a00640 1 --2- 172.21.15.143:0/404586449 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8b4007390 0x7fe8d413c660 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:58.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.105+0000 7fe8d8a00640 1 -- 172.21.15.143:0/404586449 >> 172.21.15.143:0/404586449 conn(0x7fe8d40cb830 msgr2=0x7fe8d40cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:58.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.105+0000 7fe8d8a00640 1 -- 172.21.15.143:0/404586449 shutdown_connections 2024-01-16T15:00:58.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.106+0000 7fe8d8a00640 1 -- 172.21.15.143:0/404586449 wait complete. 2024-01-16T15:00:58.103 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T15:00:58.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.324+0000 7f8cf9000640 1 Processor -- start 2024-01-16T15:00:58.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.325+0000 7f8cf9000640 1 -- start start 2024-01-16T15:00:58.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.326+0000 7f8cf9000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8cf415da20 0x7f8cf40d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:58.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.326+0000 7f8cf9000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8cf40d1fd0 0x7f8cf40cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:58.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.326+0000 7f8cf9000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cf40ce0a0 con 0x7f8cf40cf2f0 2024-01-16T15:00:58.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.326+0000 7f8cf9000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cf40d0d70 con 0x7f8cf415da20 2024-01-16T15:00:58.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.326+0000 7f8cf9000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf40d0a90 con 0x7f8cf40d1fd0 2024-01-16T15:00:58.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.326+0000 7f8cf1e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8cf415da20 0x7f8cf40d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:58.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.326+0000 7f8cf1e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8cf415da20 0x7f8cf40d0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48068/0 (socket says 172.21.15.143:48068) 2024-01-16T15:00:58.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.326+0000 7f8cf1e00640 1 -- 172.21.15.143:0/4030646922 learned_addr learned my addr 172.21.15.143:0/4030646922 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:58.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.327+0000 7f8cf1e00640 1 -- 172.21.15.143:0/4030646922 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8cf40d1fd0 msgr2=0x7f8cf40cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:58.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.327+0000 7f8cf1e00640 1 --2- 172.21.15.143:0/4030646922 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8cf40d1fd0 0x7f8cf40cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:58.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.327+0000 7f8cf1e00640 1 -- 172.21.15.143:0/4030646922 >> v1:172.21.15.204:6789/0 conn(0x7f8cf40cf2f0 legacy=0x7f8cf406ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:58.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.327+0000 7f8cf1e00640 1 -- 172.21.15.143:0/4030646922 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf40ce2d0 con 0x7f8cf415da20 2024-01-16T15:00:58.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.328+0000 7f8cf1e00640 1 --2- 172.21.15.143:0/4030646922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cf415da20 0x7f8cf40d0480 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4009870 tx=0x7f8ce402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c4a2925e0b119a server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:58.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.328+0000 7f8cf0a00640 1 -- 172.21.15.143:0/4030646922 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce4031640 con 0x7f8cf415da20 2024-01-16T15:00:58.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.328+0000 7f8cf0a00640 1 -- 172.21.15.143:0/4030646922 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ce40317a0 con 0x7f8cf415da20 2024-01-16T15:00:58.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.328+0000 7f8cf0a00640 1 -- 172.21.15.143:0/4030646922 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce4031b30 con 0x7f8cf415da20 2024-01-16T15:00:58.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.329+0000 7f8cf9000640 1 -- 172.21.15.143:0/4030646922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cf415da20 msgr2=0x7f8cf40d0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:58.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.329+0000 7f8cf9000640 1 --2- 172.21.15.143:0/4030646922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cf415da20 0x7f8cf40d0480 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4009870 tx=0x7f8ce402b8f0 comp rx=0 tx=0).stop 2024-01-16T15:00:58.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.329+0000 7f8cf9000640 1 -- 172.21.15.143:0/4030646922 shutdown_connections 2024-01-16T15:00:58.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.329+0000 7f8cf9000640 1 --2- 172.21.15.143:0/4030646922 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8cf40d1fd0 0x7f8cf40cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:58.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.329+0000 7f8cf9000640 1 --2- 172.21.15.143:0/4030646922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cf415da20 0x7f8cf40d0480 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:58.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.329+0000 7f8cf9000640 1 -- 172.21.15.143:0/4030646922 >> 172.21.15.143:0/4030646922 conn(0x7f8cf409c7e0 msgr2=0x7f8cf408af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:58.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.329+0000 7f8cf9000640 1 -- 172.21.15.143:0/4030646922 shutdown_connections 2024-01-16T15:00:58.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.329+0000 7f8cf9000640 1 -- 172.21.15.143:0/4030646922 wait complete. 2024-01-16T15:00:58.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.331+0000 7f8cf9000640 1 Processor -- start 2024-01-16T15:00:58.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.331+0000 7f8cf9000640 1 -- start start 2024-01-16T15:00:58.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.332+0000 7f8cf9000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cf40d1fd0 0x7f8cf415cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:58.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.332+0000 7f8cf9000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cf415da20 0x7f8cf415f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:58.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.332+0000 7f8cf9000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8cf415fd60 0x7f8cf40c7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:58.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.332+0000 7f8cf9000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf4160210 con 0x7f8cf40d1fd0 2024-01-16T15:00:58.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.332+0000 7f8cf9000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf40b6190 con 0x7f8cf415da20 2024-01-16T15:00:58.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.332+0000 7f8cf9000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf40b6470 con 0x7f8cf415fd60 2024-01-16T15:00:58.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.332+0000 7f8cf1e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cf40d1fd0 0x7f8cf415cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:58.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.332+0000 7f8cf1e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cf40d1fd0 0x7f8cf415cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48070/0 (socket says 172.21.15.143:48070) 2024-01-16T15:00:58.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.332+0000 7f8cf1e00640 1 -- 172.21.15.143:0/4000744226 learned_addr learned my addr 172.21.15.143:0/4000744226 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:58.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.332+0000 7f8cf1e00640 1 -- 172.21.15.143:0/4000744226 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8cf415fd60 msgr2=0x7f8cf40c7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:58.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.332+0000 7f8cf1e00640 1 --2- 172.21.15.143:0/4000744226 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8cf415fd60 0x7f8cf40c7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:58.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.332+0000 7f8cf1e00640 1 -- 172.21.15.143:0/4000744226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cf415da20 msgr2=0x7f8cf415f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:58.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.332+0000 7f8cf1e00640 1 --2- 172.21.15.143:0/4000744226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cf415da20 0x7f8cf415f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:58.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.332+0000 7f8cf1e00640 1 -- 172.21.15.143:0/4000744226 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce40094d0 con 0x7f8cf40d1fd0 2024-01-16T15:00:58.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.333+0000 7f8cf1e00640 1 --2- 172.21.15.143:0/4000744226 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cf40d1fd0 0x7f8cf415cb40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4009870 tx=0x7f8ce40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:58.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.333+0000 7f8cea000640 1 -- 172.21.15.143:0/4000744226 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce403ac60 con 0x7f8cf40d1fd0 2024-01-16T15:00:58.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.333+0000 7f8cea000640 1 -- 172.21.15.143:0/4000744226 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ce403adc0 con 0x7f8cf40d1fd0 2024-01-16T15:00:58.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.333+0000 7f8cea000640 1 -- 172.21.15.143:0/4000744226 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce4044610 con 0x7f8cf40d1fd0 2024-01-16T15:00:58.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.333+0000 7f8cf9000640 1 -- 172.21.15.143:0/4000744226 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cf40b6750 con 0x7f8cf40d1fd0 2024-01-16T15:00:58.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.333+0000 7f8cf9000640 1 -- 172.21.15.143:0/4000744226 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cf40c8490 con 0x7f8cf40d1fd0 2024-01-16T15:00:58.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.334+0000 7f8cf9000640 1 -- 172.21.15.143:0/4000744226 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cf40d0260 con 0x7f8cf40d1fd0 2024-01-16T15:00:58.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.335+0000 7f8cea000640 1 -- 172.21.15.143:0/4000744226 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f8ce402f050 con 0x7f8cf40d1fd0 2024-01-16T15:00:58.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.336+0000 7f8cea000640 1 --2- 172.21.15.143:0/4000744226 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f8ccc06b2c0 0x7f8ccc06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:58.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.336+0000 7f8cea000640 1 -- 172.21.15.143:0/4000744226 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f8ce40abc80 con 0x7f8cf40d1fd0 2024-01-16T15:00:58.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.336+0000 7f8cf1400640 1 --2- 172.21.15.143:0/4000744226 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f8ccc06b2c0 0x7f8ccc06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:58.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.337+0000 7f8cf1400640 1 --2- 172.21.15.143:0/4000744226 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f8ccc06b2c0 0x7f8ccc06d780 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8cf40d1a10 tx=0x7f8cdc009250 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:58.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.340+0000 7f8cea000640 1 -- 172.21.15.143:0/4000744226 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8ce40769d0 con 0x7f8cf40d1fd0 2024-01-16T15:00:58.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.530+0000 7f8cf9000640 1 -- 172.21.15.143:0/4000744226 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8cf40d2410 con 0x7f8cf40d1fd0 2024-01-16T15:00:58.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.532+0000 7f8cea000640 1 -- 172.21.15.143:0/4000744226 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16814 (secure 0 0 0) 0x7f8cf40d2410 con 0x7f8cf40d1fd0 2024-01-16T15:00:58.513 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:00:58.513 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":22,"fsid":"c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0","created":"2024-01-16T14:59:38.532601+0000","modified":"2024-01-16T15:00:45.626415+0000","last_up_change":"2024-01-16T14:59:50.983699+0000","last_in_change":"2024-01-16T14:59:42.056297+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:59:51.727720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T15:00:01.010472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T15:00:02.041228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f4c8891d-ab29-472a-907d-8446f20f7012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6801","nonce":1569293626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6805","nonce":1569293626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6814","nonce":1569293626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6809","nonce":1569293626}]},"public_addr":"172.21.15.143:6801/1569293626","cluster_addr":"172.21.15.143:6805/1569293626","heartbeat_back_addr":"172.21.15.143:6814/1569293626","heartbeat_front_addr":"172.21.15.143:6809/1569293626","state":["exists","up"]},{"osd":1,"uuid":"c4a2788d-8b89-4992-82b2-7f4b9debd509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6804","nonce":1276841168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6808","nonce":1276841168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6813","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6815","nonce":1276841168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6811","nonce":1276841168}]},"public_addr":"172.21.15.143:6804/1276841168","cluster_addr":"172.21.15.143:6808/1276841168","heartbeat_back_addr":"172.21.15.143:6815/1276841168","heartbeat_front_addr":"172.21.15.143:6811/1276841168","state":["exists","up"]},{"osd":2,"uuid":"64645c1d-31b2-4794-b0f2-7ccf8df30393","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6817","nonce":1217670170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6819","nonce":1217670170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6823","nonce":1217670170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6821","nonce":1217670170}]},"public_addr":"172.21.15.143:6817/1217670170","cluster_addr":"172.21.15.143:6819/1217670170","heartbeat_back_addr":"172.21.15.143:6823/1217670170","heartbeat_front_addr":"172.21.15.143:6821/1217670170","state":["exists","up"]},{"osd":3,"uuid":"73674129-a84e-4d31-8247-7d0179ae6df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6825","nonce":3414188012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6827","nonce":3414188012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6831","nonce":3414188012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6829","nonce":3414188012}]},"public_addr":"172.21.15.143:6825/3414188012","cluster_addr":"172.21.15.143:6827/3414188012","heartbeat_back_addr":"172.21.15.143:6831/3414188012","heartbeat_front_addr":"172.21.15.143:6829/3414188012","state":["exists","up"]},{"osd":4,"uuid":"cea28c9b-2f1c-4087-b415-9960789131d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6811","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6812","nonce":1070255716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6816","nonce":1070255716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6823","nonce":1070255716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6820","nonce":1070255716}]},"public_addr":"172.21.15.204:6812/1070255716","cluster_addr":"172.21.15.204:6816/1070255716","heartbeat_back_addr":"172.21.15.204:6823/1070255716","heartbeat_front_addr":"172.21.15.204:6820/1070255716","state":["exists","up"]},{"osd":5,"uuid":"33b63560-5574-4e78-983f-b398d67a2cb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6809","nonce":3605417837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6813","nonce":3605417837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6819","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6821","nonce":3605417837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6817","nonce":3605417837}]},"public_addr":"172.21.15.204:6809/3605417837","cluster_addr":"172.21.15.204:6813/3605417837","heartbeat_back_addr":"172.21.15.204:6821/3605417837","heartbeat_front_addr":"172.21.15.204:6817/3605417837","state":["exists","up"]},{"osd":6,"uuid":"df397427-8940-4eee-9c79-eece669d735d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6801","nonce":1085000534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6803","nonce":1085000534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6808","nonce":1085000534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6805","nonce":1085000534}]},"public_addr":"172.21.15.204:6801/1085000534","cluster_addr":"172.21.15.204:6803/1085000534","heartbeat_back_addr":"172.21.15.204:6808/1085000534","heartbeat_front_addr":"172.21.15.204:6805/1085000534","state":["exists","up"]},{"osd":7,"uuid":"a2e5a15c-8312-405a-ac02-f7e1fc9df2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6825","nonce":1337220609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6827","nonce":1337220609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6831","nonce":1337220609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6829","nonce":1337220609}]},"public_addr":"172.21.15.204:6825/1337220609","cluster_addr":"172.21.15.204:6827/1337220609","heartbeat_back_addr":"172.21.15.204:6831/1337220609","heartbeat_front_addr":"172.21.15.204:6829/1337220609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:49.984507+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.395306+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.632998+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.373627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.342920+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.438366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.331926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.406657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T15:00:58.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.534+0000 7f8cf9000640 1 -- 172.21.15.143:0/4000744226 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f8ccc06b2c0 msgr2=0x7f8ccc06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:58.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.534+0000 7f8cf9000640 1 --2- 172.21.15.143:0/4000744226 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f8ccc06b2c0 0x7f8ccc06d780 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8cf40d1a10 tx=0x7f8cdc009250 comp rx=0 tx=0).stop 2024-01-16T15:00:58.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.534+0000 7f8cf9000640 1 -- 172.21.15.143:0/4000744226 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cf40d1fd0 msgr2=0x7f8cf415cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:58.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.534+0000 7f8cf9000640 1 --2- 172.21.15.143:0/4000744226 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cf40d1fd0 0x7f8cf415cb40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4009870 tx=0x7f8ce40046c0 comp rx=0 tx=0).stop 2024-01-16T15:00:58.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.535+0000 7f8cf9000640 1 -- 172.21.15.143:0/4000744226 shutdown_connections 2024-01-16T15:00:58.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.535+0000 7f8cf9000640 1 --2- 172.21.15.143:0/4000744226 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f8ccc06b2c0 0x7f8ccc06d780 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:58.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.535+0000 7f8cf9000640 1 --2- 172.21.15.143:0/4000744226 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8cf415fd60 0x7f8cf40c7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:58.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.535+0000 7f8cf9000640 1 --2- 172.21.15.143:0/4000744226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cf415da20 0x7f8cf415f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:58.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.535+0000 7f8cf9000640 1 --2- 172.21.15.143:0/4000744226 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cf40d1fd0 0x7f8cf415cb40 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:58.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.535+0000 7f8cf9000640 1 -- 172.21.15.143:0/4000744226 >> 172.21.15.143:0/4000744226 conn(0x7f8cf409c7e0 msgr2=0x7f8cf40b2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:58.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.535+0000 7f8cf9000640 1 -- 172.21.15.143:0/4000744226 shutdown_connections 2024-01-16T15:00:58.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.535+0000 7f8cf9000640 1 -- 172.21.15.143:0/4000744226 wait complete. 2024-01-16T15:00:58.535 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-16T15:00:58.535 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-16T15:00:58.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.760+0000 7f08a6000640 1 Processor -- start 2024-01-16T15:00:58.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.761+0000 7f08a6000640 1 -- start start 2024-01-16T15:00:58.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.762+0000 7f08a6000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f08a0157c70 0x7f08a0161df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:58.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.762+0000 7f08a6000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f08a015e390 0x7f08a00cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:58.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.762+0000 7f08a6000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08a00ce0b0 con 0x7f08a00cf450 2024-01-16T15:00:58.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.762+0000 7f08a6000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08a00cfe40 con 0x7f08a0157c70 2024-01-16T15:00:58.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.762+0000 7f08a6000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08a0161960 con 0x7f08a015e390 2024-01-16T15:00:58.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.762+0000 7f089f400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f08a0157c70 0x7f08a0161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:58.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.762+0000 7f089f400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f08a0157c70 0x7f08a0161df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48072/0 (socket says 172.21.15.143:48072) 2024-01-16T15:00:58.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.762+0000 7f089f400640 1 -- 172.21.15.143:0/3546781448 learned_addr learned my addr 172.21.15.143:0/3546781448 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:58.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.763+0000 7f089f400640 1 -- 172.21.15.143:0/3546781448 >> v2:172.21.15.204:3300/0 conn(0x7f08a015e390 msgr2=0x7f08a00cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:58.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.763+0000 7f089f400640 1 --2- 172.21.15.143:0/3546781448 >> v2:172.21.15.204:3300/0 conn(0x7f08a015e390 0x7f08a00cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:58.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.763+0000 7f089f400640 1 -- 172.21.15.143:0/3546781448 >> v1:172.21.15.204:6789/0 conn(0x7f08a00cf450 legacy=0x7f08a015dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:58.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.763+0000 7f089f400640 1 -- 172.21.15.143:0/3546781448 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a015e8d0 con 0x7f08a0157c70 2024-01-16T15:00:58.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.764+0000 7f089f400640 1 --2- 172.21.15.143:0/3546781448 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08a0157c70 0x7f08a0161df0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f08a00d0100 tx=0x7f089002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac850c39f3ec381 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:58.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.765+0000 7f089e000640 1 -- 172.21.15.143:0/3546781448 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08900317e0 con 0x7f08a0157c70 2024-01-16T15:00:58.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.765+0000 7f089e000640 1 -- 172.21.15.143:0/3546781448 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0890031940 con 0x7f08a0157c70 2024-01-16T15:00:58.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.765+0000 7f089e000640 1 -- 172.21.15.143:0/3546781448 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0890031cd0 con 0x7f08a0157c70 2024-01-16T15:00:58.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.765+0000 7f08a6000640 1 -- 172.21.15.143:0/3546781448 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08a0157c70 msgr2=0x7f08a0161df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:58.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.765+0000 7f08a6000640 1 --2- 172.21.15.143:0/3546781448 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08a0157c70 0x7f08a0161df0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f08a00d0100 tx=0x7f089002b880 comp rx=0 tx=0).stop 2024-01-16T15:00:58.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.765+0000 7f08a6000640 1 -- 172.21.15.143:0/3546781448 shutdown_connections 2024-01-16T15:00:58.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.765+0000 7f08a6000640 1 --2- 172.21.15.143:0/3546781448 >> v2:172.21.15.204:3300/0 conn(0x7f08a015e390 0x7f08a00cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:58.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.765+0000 7f08a6000640 1 --2- 172.21.15.143:0/3546781448 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08a0157c70 0x7f08a0161df0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:58.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.765+0000 7f08a6000640 1 -- 172.21.15.143:0/3546781448 >> 172.21.15.143:0/3546781448 conn(0x7f08a006a920 msgr2=0x7f08a006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:58.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.765+0000 7f08a6000640 1 -- 172.21.15.143:0/3546781448 shutdown_connections 2024-01-16T15:00:58.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.765+0000 7f08a6000640 1 -- 172.21.15.143:0/3546781448 wait complete. 2024-01-16T15:00:58.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.767+0000 7f08a6000640 1 Processor -- start 2024-01-16T15:00:58.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.767+0000 7f08a6000640 1 -- start start 2024-01-16T15:00:58.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.768+0000 7f08a6000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08a00cf450 0x7f08a0098920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:58.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.768+0000 7f08a6000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08a0157c70 0x7f08a014d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:58.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.768+0000 7f08a6000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08a015e390 0x7f08a006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:58.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.768+0000 7f08a6000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a014fec0 con 0x7f08a0157c70 2024-01-16T15:00:58.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.768+0000 7f08a6000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a0098e60 con 0x7f08a015e390 2024-01-16T15:00:58.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.768+0000 7f08a6000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08a014f590 con 0x7f08a00cf450 2024-01-16T15:00:58.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.769+0000 7f089ea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08a0157c70 0x7f08a014d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:58.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.769+0000 7f089ea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08a0157c70 0x7f08a014d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48086/0 (socket says 172.21.15.143:48086) 2024-01-16T15:00:58.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.769+0000 7f089ea00640 1 -- 172.21.15.143:0/1641074290 learned_addr learned my addr 172.21.15.143:0/1641074290 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:58.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.769+0000 7f089f400640 1 --2- 172.21.15.143:0/1641074290 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08a00cf450 0x7f08a0098920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:58.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.769+0000 7f089fe00640 1 --2- 172.21.15.143:0/1641074290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08a015e390 0x7f08a006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:58.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.769+0000 7f089ea00640 1 -- 172.21.15.143:0/1641074290 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08a00cf450 msgr2=0x7f08a0098920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:58.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.769+0000 7f089ea00640 1 --2- 172.21.15.143:0/1641074290 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08a00cf450 0x7f08a0098920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:58.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.769+0000 7f089ea00640 1 -- 172.21.15.143:0/1641074290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08a015e390 msgr2=0x7f08a006ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:58.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.769+0000 7f089ea00640 1 --2- 172.21.15.143:0/1641074290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08a015e390 0x7f08a006ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:58.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.769+0000 7f089ea00640 1 -- 172.21.15.143:0/1641074290 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08900094d0 con 0x7f08a0157c70 2024-01-16T15:00:58.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.769+0000 7f089ea00640 1 --2- 172.21.15.143:0/1641074290 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08a0157c70 0x7f08a014d450 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f088800ba50 tx=0x7f088800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:58.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.772+0000 7f0897400640 1 -- 172.21.15.143:0/1641074290 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088800cc10 con 0x7f08a0157c70 2024-01-16T15:00:58.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.772+0000 7f0897400640 1 -- 172.21.15.143:0/1641074290 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f088800cd70 con 0x7f08a0157c70 2024-01-16T15:00:58.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.772+0000 7f0897400640 1 -- 172.21.15.143:0/1641074290 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0888014680 con 0x7f08a0157c70 2024-01-16T15:00:58.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.772+0000 7f08a6000640 1 -- 172.21.15.143:0/1641074290 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08a014f870 con 0x7f08a0157c70 2024-01-16T15:00:58.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.772+0000 7f08a6000640 1 -- 172.21.15.143:0/1641074290 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08a014f1b0 con 0x7f08a0157c70 2024-01-16T15:00:58.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.773+0000 7f08a6000640 1 -- 172.21.15.143:0/1641074290 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08a0150030 con 0x7f08a0157c70 2024-01-16T15:00:58.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.774+0000 7f0897400640 1 -- 172.21.15.143:0/1641074290 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f088801d3b0 con 0x7f08a0157c70 2024-01-16T15:00:58.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.775+0000 7f0897400640 1 --2- 172.21.15.143:0/1641074290 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f087806b2c0 0x7f087806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:58.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.775+0000 7f0897400640 1 -- 172.21.15.143:0/1641074290 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f0888089c00 con 0x7f08a0157c70 2024-01-16T15:00:58.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.775+0000 7f089f400640 1 --2- 172.21.15.143:0/1641074290 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f087806b2c0 0x7f087806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:58.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.776+0000 7f089f400640 1 --2- 172.21.15.143:0/1641074290 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f087806b2c0 0x7f087806d780 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0890036000 tx=0x7f089003d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:58.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:58.781+0000 7f0897400640 1 -- 172.21.15.143:0/1641074290 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0888090080 con 0x7f08a0157c70 2024-01-16T15:00:58.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.013+0000 7f08a6000640 1 -- 172.21.15.143:0/1641074290 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08a0067470 con 0x7f08a0157c70 2024-01-16T15:00:58.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.014+0000 7f0897400640 1 -- 172.21.15.143:0/1641074290 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4916 (secure 0 0 0) 0x7f0888019020 con 0x7f08a0157c70 2024-01-16T15:00:58.995 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:00:58.995 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3599169408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3599169408},{"type":"v1","addr":"172.21.15.204:6834","nonce":3599169408}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/859082411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":859082411},{"type":"v1","addr":"172.21.15.204:6835","nonce":859082411}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6836/1564738611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1564738611},{"type":"v1","addr":"172.21.15.143:6836","nonce":1564738611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-16T15:00:03.050082+0000","modified":"2024-01-16T15:00:06.109664+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/3120706283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6835","nonce":3120706283},{"type":"v1","addr":"172.21.15.143:6837","nonce":3120706283}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-16T15:00:58.995 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 8 2024-01-16T15:00:58.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.016+0000 7f08a6000640 1 -- 172.21.15.143:0/1641074290 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f087806b2c0 msgr2=0x7f087806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:58.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.016+0000 7f08a6000640 1 --2- 172.21.15.143:0/1641074290 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f087806b2c0 0x7f087806d780 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0890036000 tx=0x7f089003d040 comp rx=0 tx=0).stop 2024-01-16T15:00:58.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.016+0000 7f08a6000640 1 -- 172.21.15.143:0/1641074290 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08a0157c70 msgr2=0x7f08a014d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:58.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.016+0000 7f08a6000640 1 --2- 172.21.15.143:0/1641074290 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08a0157c70 0x7f08a014d450 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f088800ba50 tx=0x7f088800bf10 comp rx=0 tx=0).stop 2024-01-16T15:00:58.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.017+0000 7f08a6000640 1 -- 172.21.15.143:0/1641074290 shutdown_connections 2024-01-16T15:00:58.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.017+0000 7f08a6000640 1 --2- 172.21.15.143:0/1641074290 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f087806b2c0 0x7f087806d780 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:58.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.017+0000 7f08a6000640 1 --2- 172.21.15.143:0/1641074290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08a015e390 0x7f08a006ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:58.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.017+0000 7f08a6000640 1 --2- 172.21.15.143:0/1641074290 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08a0157c70 0x7f08a014d450 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:58.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.017+0000 7f08a6000640 1 --2- 172.21.15.143:0/1641074290 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08a00cf450 0x7f08a0098920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:58.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.017+0000 7f08a6000640 1 -- 172.21.15.143:0/1641074290 >> 172.21.15.143:0/1641074290 conn(0x7f08a006a920 msgr2=0x7f08a01568d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:58.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.017+0000 7f08a6000640 1 -- 172.21.15.143:0/1641074290 shutdown_connections 2024-01-16T15:00:58.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.017+0000 7f08a6000640 1 -- 172.21.15.143:0/1641074290 wait complete. 2024-01-16T15:00:59.017 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-16T15:00:59.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.236+0000 7f8daf200640 1 Processor -- start 2024-01-16T15:00:59.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.236+0000 7f8daf200640 1 -- start start 2024-01-16T15:00:59.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.238+0000 7f8daf200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8da815c7f0 0x7f8da806db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:59.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.238+0000 7f8daf200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8da80cf450 0x7f8da80cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:59.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.238+0000 7f8daf200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8da80cece0 con 0x7f8da80cdfb0 2024-01-16T15:00:59.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.238+0000 7f8daf200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8da80cfe40 con 0x7f8da80cf450 2024-01-16T15:00:59.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.238+0000 7f8daf200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8da80ceaf0 con 0x7f8da815c7f0 2024-01-16T15:00:59.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.238+0000 7f8da7400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8da80cf450 0x7f8da80cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:59.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.238+0000 7f8da7400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8da80cf450 0x7f8da80cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48100/0 (socket says 172.21.15.143:48100) 2024-01-16T15:00:59.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.238+0000 7f8da7400640 1 -- 172.21.15.143:0/2259520744 learned_addr learned my addr 172.21.15.143:0/2259520744 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:59.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.239+0000 7f8da7e00640 1 --2- 172.21.15.143:0/2259520744 >> v2:172.21.15.204:3300/0 conn(0x7f8da815c7f0 0x7f8da806db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:59.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.239+0000 7f8da7400640 1 -- 172.21.15.143:0/2259520744 >> v2:172.21.15.204:3300/0 conn(0x7f8da815c7f0 msgr2=0x7f8da806db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:59.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.239+0000 7f8da7400640 1 --2- 172.21.15.143:0/2259520744 >> v2:172.21.15.204:3300/0 conn(0x7f8da815c7f0 0x7f8da806db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:59.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.239+0000 7f8da7400640 1 -- 172.21.15.143:0/2259520744 >> v1:172.21.15.204:6789/0 conn(0x7f8da80cdfb0 legacy=0x7f8da80ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:59.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.239+0000 7f8da7400640 1 -- 172.21.15.143:0/2259520744 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da806d040 con 0x7f8da80cf450 2024-01-16T15:00:59.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.240+0000 7f8da7400640 1 --2- 172.21.15.143:0/2259520744 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8da80cf450 0x7f8da80cf900 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8da80d0130 tx=0x7f8d9002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e660dee82a14e582 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:59.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.240+0000 7f8da6a00640 1 -- 172.21.15.143:0/2259520744 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d90030860 con 0x7f8da80cf450 2024-01-16T15:00:59.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.240+0000 7f8da6a00640 1 -- 172.21.15.143:0/2259520744 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d900309c0 con 0x7f8da80cf450 2024-01-16T15:00:59.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.240+0000 7f8da6a00640 1 -- 172.21.15.143:0/2259520744 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d90030d50 con 0x7f8da80cf450 2024-01-16T15:00:59.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.241+0000 7f8daf200640 1 -- 172.21.15.143:0/2259520744 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8da80cf450 msgr2=0x7f8da80cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:59.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.241+0000 7f8daf200640 1 --2- 172.21.15.143:0/2259520744 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8da80cf450 0x7f8da80cf900 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8da80d0130 tx=0x7f8d9002b9a0 comp rx=0 tx=0).stop 2024-01-16T15:00:59.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.241+0000 7f8daf200640 1 -- 172.21.15.143:0/2259520744 shutdown_connections 2024-01-16T15:00:59.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.241+0000 7f8daf200640 1 --2- 172.21.15.143:0/2259520744 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8da80cf450 0x7f8da80cf900 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:59.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.241+0000 7f8daf200640 1 --2- 172.21.15.143:0/2259520744 >> v2:172.21.15.204:3300/0 conn(0x7f8da815c7f0 0x7f8da806db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:59.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.241+0000 7f8daf200640 1 -- 172.21.15.143:0/2259520744 >> 172.21.15.143:0/2259520744 conn(0x7f8da806a920 msgr2=0x7f8da806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:59.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.241+0000 7f8daf200640 1 -- 172.21.15.143:0/2259520744 shutdown_connections 2024-01-16T15:00:59.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.241+0000 7f8daf200640 1 -- 172.21.15.143:0/2259520744 wait complete. 2024-01-16T15:00:59.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.243+0000 7f8daf200640 1 Processor -- start 2024-01-16T15:00:59.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.243+0000 7f8daf200640 1 -- start start 2024-01-16T15:00:59.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.244+0000 7f8daf200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8da80cdfb0 0x7f8da80ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:59.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.244+0000 7f8daf200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8da80cf450 0x7f8da80adeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:59.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.244+0000 7f8daf200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8da815c7f0 0x7f8da80abfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:59.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.244+0000 7f8daf200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da80ad3b0 con 0x7f8da80cdfb0 2024-01-16T15:00:59.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.244+0000 7f8daf200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da80ac500 con 0x7f8da80cf450 2024-01-16T15:00:59.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.244+0000 7f8daf200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8da80ac730 con 0x7f8da815c7f0 2024-01-16T15:00:59.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.244+0000 7f8da7e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8da80cdfb0 0x7f8da80ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:59.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.244+0000 7f8da7e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8da80cdfb0 0x7f8da80ad7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48116/0 (socket says 172.21.15.143:48116) 2024-01-16T15:00:59.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.244+0000 7f8da7e00640 1 -- 172.21.15.143:0/2620849111 learned_addr learned my addr 172.21.15.143:0/2620849111 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:59.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.245+0000 7f8da7400640 1 --2- 172.21.15.143:0/2620849111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8da80cf450 0x7f8da80adeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:59.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.245+0000 7f8da7e00640 1 -- 172.21.15.143:0/2620849111 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8da815c7f0 msgr2=0x7f8da80abfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:59.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.245+0000 7f8da7e00640 1 --2- 172.21.15.143:0/2620849111 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8da815c7f0 0x7f8da80abfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:59.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.245+0000 7f8da7e00640 1 -- 172.21.15.143:0/2620849111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8da80cf450 msgr2=0x7f8da80adeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:59.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.245+0000 7f8da7e00640 1 --2- 172.21.15.143:0/2620849111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8da80cf450 0x7f8da80adeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:59.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.245+0000 7f8da7e00640 1 -- 172.21.15.143:0/2620849111 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d900094d0 con 0x7f8da80cdfb0 2024-01-16T15:00:59.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.245+0000 7f8da7400640 1 --2- 172.21.15.143:0/2620849111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8da80cf450 0x7f8da80adeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:00:59.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.245+0000 7f8da7e00640 1 --2- 172.21.15.143:0/2620849111 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8da80cdfb0 0x7f8da80ad7d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8d9800d5f0 tx=0x7f8d9800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:59.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.245+0000 7f8d9fe00640 1 -- 172.21.15.143:0/2620849111 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d980125f0 con 0x7f8da80cdfb0 2024-01-16T15:00:59.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.245+0000 7f8d9fe00640 1 -- 172.21.15.143:0/2620849111 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d98012750 con 0x7f8da80cdfb0 2024-01-16T15:00:59.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.245+0000 7f8d9fe00640 1 -- 172.21.15.143:0/2620849111 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d98012b00 con 0x7f8da80cdfb0 2024-01-16T15:00:59.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.245+0000 7f8daf200640 1 -- 172.21.15.143:0/2620849111 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8da80aca10 con 0x7f8da80cdfb0 2024-01-16T15:00:59.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.246+0000 7f8daf200640 1 -- 172.21.15.143:0/2620849111 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da80acfd0 con 0x7f8da80cdfb0 2024-01-16T15:00:59.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.247+0000 7f8daf200640 1 -- 172.21.15.143:0/2620849111 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8da8067470 con 0x7f8da80cdfb0 2024-01-16T15:00:59.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.247+0000 7f8d9fe00640 1 -- 172.21.15.143:0/2620849111 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f8d98012c60 con 0x7f8da80cdfb0 2024-01-16T15:00:59.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.248+0000 7f8d9fe00640 1 --2- 172.21.15.143:0/2620849111 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f8d8006b2c0 0x7f8d8006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:59.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.248+0000 7f8d9fe00640 1 -- 172.21.15.143:0/2620849111 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f8d9808a120 con 0x7f8da80cdfb0 2024-01-16T15:00:59.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.249+0000 7f8da7400640 1 --2- 172.21.15.143:0/2620849111 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f8d8006b2c0 0x7f8d8006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:59.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.249+0000 7f8da7400640 1 --2- 172.21.15.143:0/2620849111 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f8d8006b2c0 0x7f8d8006d780 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8d90036000 tx=0x7f8d9003b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:59.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.253+0000 7f8d9fe00640 1 -- 172.21.15.143:0/2620849111 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8d98054f10 con 0x7f8da80cdfb0 2024-01-16T15:00:59.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.477+0000 7f8daf200640 1 -- 172.21.15.143:0/2620849111 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8da80afd50 con 0x7f8da80cdfb0 2024-01-16T15:00:59.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.479+0000 7f8d9fe00640 1 -- 172.21.15.143:0/2620849111 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4916 (secure 0 0 0) 0x7f8d98058530 con 0x7f8da80cdfb0 2024-01-16T15:00:59.460 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:00:59.460 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3599169408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3599169408},{"type":"v1","addr":"172.21.15.204:6834","nonce":3599169408}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/859082411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":859082411},{"type":"v1","addr":"172.21.15.204:6835","nonce":859082411}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6836/1564738611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1564738611},{"type":"v1","addr":"172.21.15.143:6836","nonce":1564738611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-16T15:00:03.050082+0000","modified":"2024-01-16T15:00:06.109664+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/3120706283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6835","nonce":3120706283},{"type":"v1","addr":"172.21.15.143:6837","nonce":3120706283}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-16T15:00:59.460 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 8 2024-01-16T15:00:59.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.481+0000 7f8daf200640 1 -- 172.21.15.143:0/2620849111 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f8d8006b2c0 msgr2=0x7f8d8006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:59.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.481+0000 7f8daf200640 1 --2- 172.21.15.143:0/2620849111 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f8d8006b2c0 0x7f8d8006d780 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8d90036000 tx=0x7f8d9003b040 comp rx=0 tx=0).stop 2024-01-16T15:00:59.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.481+0000 7f8daf200640 1 -- 172.21.15.143:0/2620849111 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8da80cdfb0 msgr2=0x7f8da80ad7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:59.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.481+0000 7f8daf200640 1 --2- 172.21.15.143:0/2620849111 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8da80cdfb0 0x7f8da80ad7d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8d9800d5f0 tx=0x7f8d9800dab0 comp rx=0 tx=0).stop 2024-01-16T15:00:59.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.481+0000 7f8daf200640 1 -- 172.21.15.143:0/2620849111 shutdown_connections 2024-01-16T15:00:59.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.481+0000 7f8daf200640 1 --2- 172.21.15.143:0/2620849111 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f8d8006b2c0 0x7f8d8006d780 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:59.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.481+0000 7f8daf200640 1 --2- 172.21.15.143:0/2620849111 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8da815c7f0 0x7f8da80abfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:59.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.481+0000 7f8daf200640 1 --2- 172.21.15.143:0/2620849111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8da80cf450 0x7f8da80adeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:59.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.481+0000 7f8daf200640 1 --2- 172.21.15.143:0/2620849111 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8da80cdfb0 0x7f8da80ad7d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:59.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.481+0000 7f8daf200640 1 -- 172.21.15.143:0/2620849111 >> 172.21.15.143:0/2620849111 conn(0x7f8da806a920 msgr2=0x7f8da80b0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:59.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.482+0000 7f8daf200640 1 -- 172.21.15.143:0/2620849111 shutdown_connections 2024-01-16T15:00:59.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.482+0000 7f8daf200640 1 -- 172.21.15.143:0/2620849111 wait complete. 2024-01-16T15:00:59.478 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T15:00:59.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.699+0000 7f0243200640 1 Processor -- start 2024-01-16T15:00:59.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.699+0000 7f0243200640 1 -- start start 2024-01-16T15:00:59.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.701+0000 7f0243200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f023c0cf450 0x7f023c0cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:59.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.701+0000 7f0243200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f023c0cdfb0 0x7f023c0ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:59.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.701+0000 7f0243200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f023c0cece0 con 0x7f023c15c7f0 2024-01-16T15:00:59.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.701+0000 7f0243200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f023c156010 con 0x7f023c0cdfb0 2024-01-16T15:00:59.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.701+0000 7f0243200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f023c06ce30 con 0x7f023c0cf450 2024-01-16T15:00:59.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.701+0000 7f0241200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f023c0cdfb0 0x7f023c0ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:59.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.701+0000 7f023be00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f023c15c7f0 0x7f023c06db20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43408/0 (socket says 172.21.15.143:43408) 2024-01-16T15:00:59.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.701+0000 7f023be00640 1 -- 172.21.15.143:0/2157695258 learned_addr learned my addr 172.21.15.143:0/2157695258 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:00:59.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.701+0000 7f023b400640 1 --2- 172.21.15.143:0/2157695258 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f023c0cf450 0x7f023c0cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:59.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.702+0000 7f0241200640 1 -- 172.21.15.143:0/2157695258 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f023c0cf450 msgr2=0x7f023c0cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:00:59.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.702+0000 7f0241200640 1 --2- 172.21.15.143:0/2157695258 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f023c0cf450 0x7f023c0cf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:59.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.702+0000 7f0241200640 1 -- 172.21.15.143:0/2157695258 >> v1:172.21.15.143:6789/0 conn(0x7f023c15c7f0 legacy=0x7f023c06db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:59.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.702+0000 7f0241200640 1 -- 172.21.15.143:0/2157695258 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f023c06d040 con 0x7f023c0cdfb0 2024-01-16T15:00:59.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.702+0000 7f023aa00640 1 -- 172.21.15.143:0/2157695258 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2503229796 0 0) 0x7f023c0cece0 con 0x7f023c15c7f0 2024-01-16T15:00:59.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.703+0000 7f0241200640 1 --2- 172.21.15.143:0/2157695258 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f023c0cdfb0 0x7f023c0ce440 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0234009d90 tx=0x7f023402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23bdf61309d58274 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:59.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.703+0000 7f023aa00640 1 -- 172.21.15.143:0/2157695258 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023402cc50 con 0x7f023c0cdfb0 2024-01-16T15:00:59.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.703+0000 7f023aa00640 1 -- 172.21.15.143:0/2157695258 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f023402cdb0 con 0x7f023c0cdfb0 2024-01-16T15:00:59.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.703+0000 7f023aa00640 1 -- 172.21.15.143:0/2157695258 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0234034660 con 0x7f023c0cdfb0 2024-01-16T15:00:59.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.704+0000 7f0243200640 1 -- 172.21.15.143:0/2157695258 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f023c0cdfb0 msgr2=0x7f023c0ce440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:59.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.704+0000 7f0243200640 1 --2- 172.21.15.143:0/2157695258 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f023c0cdfb0 0x7f023c0ce440 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0234009d90 tx=0x7f023402bea0 comp rx=0 tx=0).stop 2024-01-16T15:00:59.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.704+0000 7f0243200640 1 -- 172.21.15.143:0/2157695258 shutdown_connections 2024-01-16T15:00:59.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.704+0000 7f0243200640 1 --2- 172.21.15.143:0/2157695258 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f023c0cdfb0 0x7f023c0ce440 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:59.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.704+0000 7f0243200640 1 --2- 172.21.15.143:0/2157695258 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f023c0cf450 0x7f023c0cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:59.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.704+0000 7f0243200640 1 -- 172.21.15.143:0/2157695258 >> 172.21.15.143:0/2157695258 conn(0x7f023c06a920 msgr2=0x7f023c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:59.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.704+0000 7f0243200640 1 -- 172.21.15.143:0/2157695258 shutdown_connections 2024-01-16T15:00:59.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.704+0000 7f0243200640 1 -- 172.21.15.143:0/2157695258 wait complete. 2024-01-16T15:00:59.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.705+0000 7f0243200640 1 Processor -- start 2024-01-16T15:00:59.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.705+0000 7f0243200640 1 -- start start 2024-01-16T15:00:59.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.706+0000 7f0243200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f023c0cdfb0 0x7f023c155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:59.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.706+0000 7f0243200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f023c0cf450 0x7f023c0abfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:59.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.706+0000 7f0243200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f023c15c7f0 0x7f023c0ac500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:59.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.706+0000 7f0243200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023c1f3410 con 0x7f023c15c7f0 2024-01-16T15:00:59.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.706+0000 7f0243200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023c0acb10 con 0x7f023c0cdfb0 2024-01-16T15:00:59.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.707+0000 7f0243200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f023c0acdf0 con 0x7f023c0cf450 2024-01-16T15:00:59.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.707+0000 7f0241200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f023c15c7f0 0x7f023c0ac500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:59.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.707+0000 7f0241200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f023c15c7f0 0x7f023c0ac500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48132/0 (socket says 172.21.15.143:48132) 2024-01-16T15:00:59.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.707+0000 7f0241200640 1 -- 172.21.15.143:0/2236197991 learned_addr learned my addr 172.21.15.143:0/2236197991 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:00:59.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.707+0000 7f0241200640 1 -- 172.21.15.143:0/2236197991 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f023c0cf450 msgr2=0x7f023c0abfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:59.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.707+0000 7f0241200640 1 --2- 172.21.15.143:0/2236197991 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f023c0cf450 0x7f023c0abfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:59.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.707+0000 7f0241200640 1 -- 172.21.15.143:0/2236197991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f023c0cdfb0 msgr2=0x7f023c155480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:59.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.707+0000 7f0241200640 1 --2- 172.21.15.143:0/2236197991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f023c0cdfb0 0x7f023c155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:59.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.707+0000 7f0241200640 1 -- 172.21.15.143:0/2236197991 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0234009a70 con 0x7f023c15c7f0 2024-01-16T15:00:59.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.707+0000 7f0241200640 1 --2- 172.21.15.143:0/2236197991 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f023c15c7f0 0x7f023c0ac500 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0234009d90 tx=0x7f0234004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:59.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.708+0000 7f0233e00640 1 -- 172.21.15.143:0/2236197991 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0234040650 con 0x7f023c15c7f0 2024-01-16T15:00:59.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.708+0000 7f0233e00640 1 -- 172.21.15.143:0/2236197991 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02340407b0 con 0x7f023c15c7f0 2024-01-16T15:00:59.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.708+0000 7f0243200640 1 -- 172.21.15.143:0/2236197991 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f023c0ad0d0 con 0x7f023c15c7f0 2024-01-16T15:00:59.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.708+0000 7f0233e00640 1 -- 172.21.15.143:0/2236197991 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0234040b60 con 0x7f023c15c7f0 2024-01-16T15:00:59.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.708+0000 7f0243200640 1 -- 172.21.15.143:0/2236197991 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f023c0ad690 con 0x7f023c15c7f0 2024-01-16T15:00:59.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.709+0000 7f0243200640 1 -- 172.21.15.143:0/2236197991 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f023c0c8360 con 0x7f023c15c7f0 2024-01-16T15:00:59.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.710+0000 7f0233e00640 1 -- 172.21.15.143:0/2236197991 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f0234040cc0 con 0x7f023c15c7f0 2024-01-16T15:00:59.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.710+0000 7f0233e00640 1 --2- 172.21.15.143:0/2236197991 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f021406b2f0 0x7f021406d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:00:59.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.711+0000 7f0233e00640 1 -- 172.21.15.143:0/2236197991 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f02340aca70 con 0x7f023c15c7f0 2024-01-16T15:00:59.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.711+0000 7f023be00640 1 --2- 172.21.15.143:0/2236197991 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f021406b2f0 0x7f021406d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:00:59.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.711+0000 7f023be00640 1 --2- 172.21.15.143:0/2236197991 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f021406b2f0 0x7f021406d7b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f022c007fd0 tx=0x7f022c007960 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:00:59.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.714+0000 7f0233e00640 1 -- 172.21.15.143:0/2236197991 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f023c0c8360 con 0x7f023c15c7f0 2024-01-16T15:00:59.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.906+0000 7f0243200640 1 -- 172.21.15.143:0/2236197991 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f023c0c8360 con 0x7f023c15c7f0 2024-01-16T15:00:59.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.908+0000 7f0233e00640 1 -- 172.21.15.143:0/2236197991 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16814 (secure 0 0 0) 0x7f023c0c8360 con 0x7f023c15c7f0 2024-01-16T15:00:59.889 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:00:59.889 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":22,"fsid":"c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0","created":"2024-01-16T14:59:38.532601+0000","modified":"2024-01-16T15:00:45.626415+0000","last_up_change":"2024-01-16T14:59:50.983699+0000","last_in_change":"2024-01-16T14:59:42.056297+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:59:51.727720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T15:00:01.010472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T15:00:02.041228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f4c8891d-ab29-472a-907d-8446f20f7012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6801","nonce":1569293626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6805","nonce":1569293626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6814","nonce":1569293626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6809","nonce":1569293626}]},"public_addr":"172.21.15.143:6801/1569293626","cluster_addr":"172.21.15.143:6805/1569293626","heartbeat_back_addr":"172.21.15.143:6814/1569293626","heartbeat_front_addr":"172.21.15.143:6809/1569293626","state":["exists","up"]},{"osd":1,"uuid":"c4a2788d-8b89-4992-82b2-7f4b9debd509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6804","nonce":1276841168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6808","nonce":1276841168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6813","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6815","nonce":1276841168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6811","nonce":1276841168}]},"public_addr":"172.21.15.143:6804/1276841168","cluster_addr":"172.21.15.143:6808/1276841168","heartbeat_back_addr":"172.21.15.143:6815/1276841168","heartbeat_front_addr":"172.21.15.143:6811/1276841168","state":["exists","up"]},{"osd":2,"uuid":"64645c1d-31b2-4794-b0f2-7ccf8df30393","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6817","nonce":1217670170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6819","nonce":1217670170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6823","nonce":1217670170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6821","nonce":1217670170}]},"public_addr":"172.21.15.143:6817/1217670170","cluster_addr":"172.21.15.143:6819/1217670170","heartbeat_back_addr":"172.21.15.143:6823/1217670170","heartbeat_front_addr":"172.21.15.143:6821/1217670170","state":["exists","up"]},{"osd":3,"uuid":"73674129-a84e-4d31-8247-7d0179ae6df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6825","nonce":3414188012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6827","nonce":3414188012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6831","nonce":3414188012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6829","nonce":3414188012}]},"public_addr":"172.21.15.143:6825/3414188012","cluster_addr":"172.21.15.143:6827/3414188012","heartbeat_back_addr":"172.21.15.143:6831/3414188012","heartbeat_front_addr":"172.21.15.143:6829/3414188012","state":["exists","up"]},{"osd":4,"uuid":"cea28c9b-2f1c-4087-b415-9960789131d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6811","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6812","nonce":1070255716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6816","nonce":1070255716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6823","nonce":1070255716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6820","nonce":1070255716}]},"public_addr":"172.21.15.204:6812/1070255716","cluster_addr":"172.21.15.204:6816/1070255716","heartbeat_back_addr":"172.21.15.204:6823/1070255716","heartbeat_front_addr":"172.21.15.204:6820/1070255716","state":["exists","up"]},{"osd":5,"uuid":"33b63560-5574-4e78-983f-b398d67a2cb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6809","nonce":3605417837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6813","nonce":3605417837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6819","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6821","nonce":3605417837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6817","nonce":3605417837}]},"public_addr":"172.21.15.204:6809/3605417837","cluster_addr":"172.21.15.204:6813/3605417837","heartbeat_back_addr":"172.21.15.204:6821/3605417837","heartbeat_front_addr":"172.21.15.204:6817/3605417837","state":["exists","up"]},{"osd":6,"uuid":"df397427-8940-4eee-9c79-eece669d735d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6801","nonce":1085000534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6803","nonce":1085000534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6808","nonce":1085000534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6805","nonce":1085000534}]},"public_addr":"172.21.15.204:6801/1085000534","cluster_addr":"172.21.15.204:6803/1085000534","heartbeat_back_addr":"172.21.15.204:6808/1085000534","heartbeat_front_addr":"172.21.15.204:6805/1085000534","state":["exists","up"]},{"osd":7,"uuid":"a2e5a15c-8312-405a-ac02-f7e1fc9df2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6825","nonce":1337220609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6827","nonce":1337220609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6831","nonce":1337220609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6829","nonce":1337220609}]},"public_addr":"172.21.15.204:6825/1337220609","cluster_addr":"172.21.15.204:6827/1337220609","heartbeat_back_addr":"172.21.15.204:6831/1337220609","heartbeat_front_addr":"172.21.15.204:6829/1337220609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:49.984507+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.395306+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.632998+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.373627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.342920+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.438366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.331926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.406657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T15:00:59.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.911+0000 7f0243200640 1 -- 172.21.15.143:0/2236197991 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f021406b2f0 msgr2=0x7f021406d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:59.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.911+0000 7f0243200640 1 --2- 172.21.15.143:0/2236197991 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f021406b2f0 0x7f021406d7b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f022c007fd0 tx=0x7f022c007960 comp rx=0 tx=0).stop 2024-01-16T15:00:59.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.911+0000 7f0243200640 1 -- 172.21.15.143:0/2236197991 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f023c15c7f0 msgr2=0x7f023c0ac500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:00:59.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.911+0000 7f0243200640 1 --2- 172.21.15.143:0/2236197991 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f023c15c7f0 0x7f023c0ac500 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0234009d90 tx=0x7f0234004a40 comp rx=0 tx=0).stop 2024-01-16T15:00:59.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.911+0000 7f0243200640 1 -- 172.21.15.143:0/2236197991 shutdown_connections 2024-01-16T15:00:59.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.911+0000 7f0243200640 1 --2- 172.21.15.143:0/2236197991 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f021406b2f0 0x7f021406d7b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:59.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.911+0000 7f0243200640 1 --2- 172.21.15.143:0/2236197991 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f023c15c7f0 0x7f023c0ac500 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:59.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.911+0000 7f0243200640 1 --2- 172.21.15.143:0/2236197991 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f023c0cf450 0x7f023c0abfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:59.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.911+0000 7f0243200640 1 --2- 172.21.15.143:0/2236197991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f023c0cdfb0 0x7f023c155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:00:59.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.911+0000 7f0243200640 1 -- 172.21.15.143:0/2236197991 >> 172.21.15.143:0/2236197991 conn(0x7f023c06a920 msgr2=0x7f023c15fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:00:59.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.911+0000 7f0243200640 1 -- 172.21.15.143:0/2236197991 shutdown_connections 2024-01-16T15:00:59.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:00:59.911+0000 7f0243200640 1 -- 172.21.15.143:0/2236197991 wait complete. 2024-01-16T15:00:59.911 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-16T15:01:00.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.136+0000 7f7a2c400640 1 Processor -- start 2024-01-16T15:01:00.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.136+0000 7f7a2c400640 1 -- start start 2024-01-16T15:01:00.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.138+0000 7f7a2c400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a2415c840 0x7f7a240cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:00.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.138+0000 7f7a2c400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7a240aed70 0x7f7a2415feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:00.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.138+0000 7f7a2c400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a240cbd10 con 0x7f7a240ce8b0 2024-01-16T15:01:00.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.138+0000 7f7a2c400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a2415bce0 con 0x7f7a240aed70 2024-01-16T15:01:00.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.138+0000 7f7a2c400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a240cbff0 con 0x7f7a2415c840 2024-01-16T15:01:00.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.138+0000 7f7a29a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f7a240ce8b0 0x7f7a240ccad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43410/0 (socket says 172.21.15.143:43410) 2024-01-16T15:01:00.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.138+0000 7f7a29a00640 1 -- 172.21.15.143:0/557555427 learned_addr learned my addr 172.21.15.143:0/557555427 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:01:00.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.139+0000 7f7a2a400640 1 --2- 172.21.15.143:0/557555427 >> v2:172.21.15.204:3300/0 conn(0x7f7a240aed70 0x7f7a2415feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:00.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.139+0000 7f7a29000640 1 --2- 172.21.15.143:0/557555427 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a2415c840 0x7f7a240cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:00.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.139+0000 7f7a23e00640 1 -- 172.21.15.143:0/557555427 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2880166933 0 0) 0x7f7a240cbd10 con 0x7f7a240ce8b0 2024-01-16T15:01:00.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.139+0000 7f7a23e00640 1 -- 172.21.15.143:0/557555427 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a04003520 con 0x7f7a240ce8b0 2024-01-16T15:01:00.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.139+0000 7f7a23e00640 1 -- 172.21.15.143:0/557555427 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3030131131 0 0) 0x7f7a04003520 con 0x7f7a240ce8b0 2024-01-16T15:01:00.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.139+0000 7f7a23e00640 1 -- 172.21.15.143:0/557555427 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a2415c840 msgr2=0x7f7a240cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:00.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.139+0000 7f7a23e00640 1 --2- 172.21.15.143:0/557555427 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a2415c840 0x7f7a240cec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:00.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.139+0000 7f7a23e00640 1 -- 172.21.15.143:0/557555427 >> v2:172.21.15.204:3300/0 conn(0x7f7a240aed70 msgr2=0x7f7a2415feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:00.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.139+0000 7f7a23e00640 1 --2- 172.21.15.143:0/557555427 >> v2:172.21.15.204:3300/0 conn(0x7f7a240aed70 0x7f7a2415feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:00.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.139+0000 7f7a23e00640 1 -- 172.21.15.143:0/557555427 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a240cc2d0 con 0x7f7a240ce8b0 2024-01-16T15:01:00.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.139+0000 7f7a23e00640 1 -- 172.21.15.143:0/557555427 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7f7a14001670 con 0x7f7a240ce8b0 2024-01-16T15:01:00.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.139+0000 7f7a23e00640 1 -- 172.21.15.143:0/557555427 >> v1:172.21.15.143:6789/0 conn(0x7f7a240ce8b0 legacy=0x7f7a240ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:00.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.139+0000 7f7a23e00640 1 --2- 172.21.15.143:0/557555427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a04003970 0x7f7a04003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:00.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.139+0000 7f7a23e00640 1 --2- 172.21.15.143:0/557555427 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a2415c840 0x7f7a040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:00.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.139+0000 7f7a23e00640 1 --2- 172.21.15.143:0/557555427 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a04004d00 0x7f7a04005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:00.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.139+0000 7f7a23e00640 1 -- 172.21.15.143:0/557555427 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a240cbff0 con 0x7f7a04004d00 2024-01-16T15:01:00.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.139+0000 7f7a23e00640 1 -- 172.21.15.143:0/557555427 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0400b980 con 0x7f7a04003970 2024-01-16T15:01:00.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.139+0000 7f7a23e00640 1 -- 172.21.15.143:0/557555427 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0400bac0 con 0x7f7a2415c840 2024-01-16T15:01:00.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.140+0000 7f7a2a400640 1 --2- 172.21.15.143:0/557555427 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a04004d00 0x7f7a04005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:00.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.140+0000 7f7a29000640 1 --2- 172.21.15.143:0/557555427 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a2415c840 0x7f7a040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:00.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.140+0000 7f7a2a400640 1 -- 172.21.15.143:0/557555427 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a2415c840 msgr2=0x7f7a040047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:00.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.140+0000 7f7a2a400640 1 --2- 172.21.15.143:0/557555427 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a2415c840 0x7f7a040047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:00.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.140+0000 7f7a2a400640 1 -- 172.21.15.143:0/557555427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a04003970 msgr2=0x7f7a04003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:00.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.140+0000 7f7a2a400640 1 --2- 172.21.15.143:0/557555427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a04003970 0x7f7a04003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:00.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.140+0000 7f7a2a400640 1 -- 172.21.15.143:0/557555427 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a04003520 con 0x7f7a04004d00 2024-01-16T15:01:00.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.141+0000 7f7a2a400640 1 --2- 172.21.15.143:0/557555427 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a04004d00 0x7f7a04005180 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c009fc0 tx=0x7f7a040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7ca2aa094444433 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:00.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.141+0000 7f7a23e00640 1 -- 172.21.15.143:0/557555427 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c02cc60 con 0x7f7a04004d00 2024-01-16T15:01:00.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.142+0000 7f7a23e00640 1 -- 172.21.15.143:0/557555427 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a1c00f040 con 0x7f7a04004d00 2024-01-16T15:01:00.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.142+0000 7f7a23e00640 1 -- 172.21.15.143:0/557555427 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c034440 con 0x7f7a04004d00 2024-01-16T15:01:00.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.142+0000 7f7a2c400640 1 -- 172.21.15.143:0/557555427 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a04004d00 msgr2=0x7f7a04005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:00.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.142+0000 7f7a2c400640 1 --2- 172.21.15.143:0/557555427 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a04004d00 0x7f7a04005180 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c009fc0 tx=0x7f7a040057f0 comp rx=0 tx=0).stop 2024-01-16T15:01:00.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.142+0000 7f7a2a400640 1 -- 172.21.15.143:0/557555427 reap_dead start 2024-01-16T15:01:00.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.142+0000 7f7a2c400640 1 -- 172.21.15.143:0/557555427 shutdown_connections 2024-01-16T15:01:00.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.142+0000 7f7a2c400640 1 -- 172.21.15.143:0/557555427 >> 172.21.15.143:0/557555427 conn(0x7f7a240cf900 msgr2=0x7f7a240cfd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:00.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.142+0000 7f7a2c400640 1 -- 172.21.15.143:0/557555427 shutdown_connections 2024-01-16T15:01:00.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.142+0000 7f7a2c400640 1 -- 172.21.15.143:0/557555427 wait complete. 2024-01-16T15:01:00.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.144+0000 7f7a2c400640 1 Processor -- start 2024-01-16T15:01:00.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.144+0000 7f7a2c400640 1 -- start start 2024-01-16T15:01:00.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.145+0000 7f7a2c400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a04004d00 0x7f7a24153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:00.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.145+0000 7f7a2c400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a24159060 0x7f7a241594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:00.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.145+0000 7f7a2c400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a240ccad0 0x7f7a240ccf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:00.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.145+0000 7f7a2c400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2415a7c0 con 0x7f7a240ccad0 2024-01-16T15:01:00.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.145+0000 7f7a2c400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a240cd4c0 con 0x7f7a04004d00 2024-01-16T15:01:00.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.145+0000 7f7a2c400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a24159a20 con 0x7f7a24159060 2024-01-16T15:01:00.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.145+0000 7f7a2a400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a240ccad0 0x7f7a240ccf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:00.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.145+0000 7f7a2a400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a240ccad0 0x7f7a240ccf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48158/0 (socket says 172.21.15.143:48158) 2024-01-16T15:01:00.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.145+0000 7f7a2a400640 1 -- 172.21.15.143:0/3325837318 learned_addr learned my addr 172.21.15.143:0/3325837318 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:00.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.146+0000 7f7a29a00640 1 --2- 172.21.15.143:0/3325837318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a04004d00 0x7f7a24153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:00.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.146+0000 7f7a2a400640 1 -- 172.21.15.143:0/3325837318 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a24159060 msgr2=0x7f7a241594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:00.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.146+0000 7f7a29000640 1 --2- 172.21.15.143:0/3325837318 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a24159060 0x7f7a241594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:00.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.146+0000 7f7a2a400640 1 --2- 172.21.15.143:0/3325837318 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a24159060 0x7f7a241594e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:00.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.146+0000 7f7a2a400640 1 -- 172.21.15.143:0/3325837318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a04004d00 msgr2=0x7f7a24153e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:00.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.146+0000 7f7a2a400640 1 --2- 172.21.15.143:0/3325837318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a04004d00 0x7f7a24153e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:00.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.146+0000 7f7a2a400640 1 -- 172.21.15.143:0/3325837318 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a2415ac70 con 0x7f7a240ccad0 2024-01-16T15:01:00.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.146+0000 7f7a29a00640 1 --2- 172.21.15.143:0/3325837318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a04004d00 0x7f7a24153e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:01:00.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.146+0000 7f7a2a400640 1 --2- 172.21.15.143:0/3325837318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a240ccad0 0x7f7a240ccf80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c00e230 tx=0x7f7a1c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:00.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.146+0000 7f7a21600640 1 -- 172.21.15.143:0/3325837318 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c046070 con 0x7f7a240ccad0 2024-01-16T15:01:00.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.146+0000 7f7a21600640 1 -- 172.21.15.143:0/3325837318 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a1c035ae0 con 0x7f7a240ccad0 2024-01-16T15:01:00.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.146+0000 7f7a2c400640 1 -- 172.21.15.143:0/3325837318 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a240abdf0 con 0x7f7a240ccad0 2024-01-16T15:01:00.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.146+0000 7f7a21600640 1 -- 172.21.15.143:0/3325837318 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c035e70 con 0x7f7a240ccad0 2024-01-16T15:01:00.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.146+0000 7f7a2c400640 1 -- 172.21.15.143:0/3325837318 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a24159f20 con 0x7f7a240ccad0 2024-01-16T15:01:00.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.148+0000 7f7a2c400640 1 -- 172.21.15.143:0/3325837318 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a2404e8b0 con 0x7f7a240ccad0 2024-01-16T15:01:00.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.149+0000 7f7a21600640 1 -- 172.21.15.143:0/3325837318 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f7a1c03b050 con 0x7f7a240ccad0 2024-01-16T15:01:00.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.150+0000 7f7a21600640 1 --2- 172.21.15.143:0/3325837318 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f79fc06b2c0 0x7f79fc06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:00.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.150+0000 7f7a21600640 1 -- 172.21.15.143:0/3325837318 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f7a1c0ab380 con 0x7f7a240ccad0 2024-01-16T15:01:00.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.150+0000 7f7a29a00640 1 --2- 172.21.15.143:0/3325837318 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f79fc06b2c0 0x7f79fc06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:00.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.151+0000 7f7a29a00640 1 --2- 172.21.15.143:0/3325837318 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f79fc06b2c0 0x7f79fc06d780 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7a14000fd0 tx=0x7f7a14009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:00.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.154+0000 7f7a21600640 1 -- 172.21.15.143:0/3325837318 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7a1c0761f0 con 0x7f7a240ccad0 2024-01-16T15:01:00.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.385+0000 7f7a2c400640 1 -- 172.21.15.143:0/3325837318 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7a240abf80 con 0x7f7a240ccad0 2024-01-16T15:01:00.366 INFO:tasks.ceph.mon.a.smithi143.stderr:2024-01-16T15:01:00.386+0000 7f2127a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-16T15:01:00.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.389+0000 7f7a21600640 1 -- 172.21.15.143:0/3325837318 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7a1c079810 con 0x7f7a240ccad0 2024-01-16T15:01:00.370 INFO:teuthology.orchestra.run.smithi143.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-16T15:01:00.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.393+0000 7f7a2c400640 1 -- 172.21.15.143:0/3325837318 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f79fc06b2c0 msgr2=0x7f79fc06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:00.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.393+0000 7f7a2c400640 1 --2- 172.21.15.143:0/3325837318 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f79fc06b2c0 0x7f79fc06d780 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7a14000fd0 tx=0x7f7a14009040 comp rx=0 tx=0).stop 2024-01-16T15:01:00.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.393+0000 7f7a2c400640 1 -- 172.21.15.143:0/3325837318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a240ccad0 msgr2=0x7f7a240ccf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:00.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.393+0000 7f7a2c400640 1 --2- 172.21.15.143:0/3325837318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a240ccad0 0x7f7a240ccf80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c00e230 tx=0x7f7a1c009750 comp rx=0 tx=0).stop 2024-01-16T15:01:00.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.394+0000 7f7a2c400640 1 -- 172.21.15.143:0/3325837318 shutdown_connections 2024-01-16T15:01:00.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.394+0000 7f7a2c400640 1 --2- 172.21.15.143:0/3325837318 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f79fc06b2c0 0x7f79fc06d780 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:00.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.394+0000 7f7a2c400640 1 --2- 172.21.15.143:0/3325837318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a240ccad0 0x7f7a240ccf80 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:00.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.394+0000 7f7a2c400640 1 --2- 172.21.15.143:0/3325837318 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a24159060 0x7f7a241594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:00.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.394+0000 7f7a2c400640 1 --2- 172.21.15.143:0/3325837318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a04004d00 0x7f7a24153e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:00.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.394+0000 7f7a2c400640 1 -- 172.21.15.143:0/3325837318 >> 172.21.15.143:0/3325837318 conn(0x7f7a240cf900 msgr2=0x7f7a240affd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:00.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.394+0000 7f7a2c400640 1 -- 172.21.15.143:0/3325837318 shutdown_connections 2024-01-16T15:01:00.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.394+0000 7f7a2c400640 1 -- 172.21.15.143:0/3325837318 wait complete. 2024-01-16T15:01:00.391 INFO:tasks.ceph.mds.a.smithi143.stderr: -7478> 2024-01-16T15:00:08.114+0000 7fae90200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-16T15:01:00.395 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-16T15:01:00.447 INFO:tasks.ceph.mds.a.smithi143.stdout:starting mds.a at 2024-01-16T15:01:00.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.599+0000 7f782b600640 1 Processor -- start 2024-01-16T15:01:00.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.599+0000 7f782b600640 1 -- start start 2024-01-16T15:01:00.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.600+0000 7f782b600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f782415b440 0x7f78240cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:00.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.600+0000 7f782b600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f78240cbf20 0x7f78240c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:00.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.600+0000 7f782b600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78240cc3a0 con 0x7f78240c57e0 2024-01-16T15:01:00.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.600+0000 7f782b600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f782415b820 con 0x7f78240cbf20 2024-01-16T15:01:00.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.600+0000 7f782b600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f782415eea0 con 0x7f782415b440 2024-01-16T15:01:00.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.601+0000 7f7823e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f78240cbf20 0x7f78240c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:00.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.601+0000 7f7829600640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f78240c57e0 0x7f78240c3ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43414/0 (socket says 172.21.15.143:43414) 2024-01-16T15:01:00.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.601+0000 7f7829600640 1 -- 172.21.15.143:0/1892481817 learned_addr learned my addr 172.21.15.143:0/1892481817 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:01:00.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.601+0000 7f7828c00640 1 --2- 172.21.15.143:0/1892481817 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f782415b440 0x7f78240cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:00.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.601+0000 7f7823e00640 1 -- 172.21.15.143:0/1892481817 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f782415b440 msgr2=0x7f78240cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:00.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.601+0000 7f7823e00640 1 --2- 172.21.15.143:0/1892481817 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f782415b440 0x7f78240cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:00.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.601+0000 7f7823e00640 1 -- 172.21.15.143:0/1892481817 >> v1:172.21.15.143:6789/0 conn(0x7f78240c57e0 legacy=0x7f78240c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:00.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.601+0000 7f7823e00640 1 -- 172.21.15.143:0/1892481817 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f782406d600 con 0x7f78240cbf20 2024-01-16T15:01:00.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.601+0000 7f7823400640 1 -- 172.21.15.143:0/1892481817 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2137080058 0 0) 0x7f781c001490 con 0x7f78240c57e0 2024-01-16T15:01:00.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.601+0000 7f7828c00640 1 --2- 172.21.15.143:0/1892481817 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f782415b440 0x7f78240cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:01:00.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.602+0000 7f7823e00640 1 --2- 172.21.15.143:0/1892481817 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78240cbf20 0x7f78240c52a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f780c0097a0 tx=0x7f780c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88893e0c326e6b10 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:00.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.602+0000 7f7823400640 1 -- 172.21.15.143:0/1892481817 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780c02e070 con 0x7f78240cbf20 2024-01-16T15:01:00.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.603+0000 7f7823400640 1 -- 172.21.15.143:0/1892481817 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f780c0315a0 con 0x7f78240cbf20 2024-01-16T15:01:00.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.603+0000 7f7823400640 1 -- 172.21.15.143:0/1892481817 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780c031930 con 0x7f78240cbf20 2024-01-16T15:01:00.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.603+0000 7f782b600640 1 -- 172.21.15.143:0/1892481817 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78240cbf20 msgr2=0x7f78240c52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:00.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.603+0000 7f782b600640 1 --2- 172.21.15.143:0/1892481817 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78240cbf20 0x7f78240c52a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f780c0097a0 tx=0x7f780c02b840 comp rx=0 tx=0).stop 2024-01-16T15:01:00.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.603+0000 7f782b600640 1 -- 172.21.15.143:0/1892481817 shutdown_connections 2024-01-16T15:01:00.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.603+0000 7f782b600640 1 --2- 172.21.15.143:0/1892481817 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78240cbf20 0x7f78240c52a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:00.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.603+0000 7f782b600640 1 --2- 172.21.15.143:0/1892481817 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f782415b440 0x7f78240cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:00.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.603+0000 7f782b600640 1 -- 172.21.15.143:0/1892481817 >> 172.21.15.143:0/1892481817 conn(0x7f78240b0c50 msgr2=0x7f78240b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:00.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.603+0000 7f782b600640 1 -- 172.21.15.143:0/1892481817 shutdown_connections 2024-01-16T15:01:00.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.603+0000 7f782b600640 1 -- 172.21.15.143:0/1892481817 wait complete. 2024-01-16T15:01:00.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.605+0000 7f782b600640 1 Processor -- start 2024-01-16T15:01:00.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.605+0000 7f782b600640 1 -- start start 2024-01-16T15:01:00.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.606+0000 7f782b600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78240c57e0 0x7f782415e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:00.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.606+0000 7f782b600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78240cbf20 0x7f782415b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:00.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.606+0000 7f782b600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f782415b440 0x7f7824159a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:00.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.606+0000 7f782b600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782409c500 con 0x7f782415b440 2024-01-16T15:01:00.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.606+0000 7f782b600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782409c7e0 con 0x7f78240cbf20 2024-01-16T15:01:00.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.606+0000 7f782b600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7824155670 con 0x7f78240c57e0 2024-01-16T15:01:00.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.606+0000 7f7829600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f782415b440 0x7f7824159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:00.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.606+0000 7f7829600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f782415b440 0x7f7824159a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48192/0 (socket says 172.21.15.143:48192) 2024-01-16T15:01:00.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.606+0000 7f7829600640 1 -- 172.21.15.143:0/1703576568 learned_addr learned my addr 172.21.15.143:0/1703576568 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:00.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.607+0000 7f7829600640 1 -- 172.21.15.143:0/1703576568 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78240c57e0 msgr2=0x7f782415e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:00.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.607+0000 7f7829600640 1 --2- 172.21.15.143:0/1703576568 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78240c57e0 0x7f782415e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:00.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.607+0000 7f7829600640 1 -- 172.21.15.143:0/1703576568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78240cbf20 msgr2=0x7f782415b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:00.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.607+0000 7f7829600640 1 --2- 172.21.15.143:0/1703576568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78240cbf20 0x7f782415b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:00.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.607+0000 7f7829600640 1 -- 172.21.15.143:0/1703576568 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f780c009400 con 0x7f782415b440 2024-01-16T15:01:00.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.607+0000 7f7829600640 1 --2- 172.21.15.143:0/1703576568 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f782415b440 0x7f7824159a80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f781c00c7d0 tx=0x7f781c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:00.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.607+0000 7f7820c00640 1 -- 172.21.15.143:0/1703576568 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c00a640 con 0x7f782415b440 2024-01-16T15:01:00.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.607+0000 7f7820c00640 1 -- 172.21.15.143:0/1703576568 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f781c00a7c0 con 0x7f782415b440 2024-01-16T15:01:00.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.607+0000 7f7820c00640 1 -- 172.21.15.143:0/1703576568 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c00ab70 con 0x7f782415b440 2024-01-16T15:01:00.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.607+0000 7f782b600640 1 -- 172.21.15.143:0/1703576568 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7824155950 con 0x7f782415b440 2024-01-16T15:01:00.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.607+0000 7f782b600640 1 -- 172.21.15.143:0/1703576568 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7824159fc0 con 0x7f782415b440 2024-01-16T15:01:00.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.608+0000 7f782b600640 1 -- 172.21.15.143:0/1703576568 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7824155e70 con 0x7f782415b440 2024-01-16T15:01:00.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.609+0000 7f7820c00640 1 -- 172.21.15.143:0/1703576568 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f781c00acd0 con 0x7f782415b440 2024-01-16T15:01:00.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.610+0000 7f7820c00640 1 --2- 172.21.15.143:0/1703576568 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f77fc06b2c0 0x7f77fc06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:00.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.610+0000 7f7820c00640 1 -- 172.21.15.143:0/1703576568 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f781c0890f0 con 0x7f782415b440 2024-01-16T15:01:00.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.610+0000 7f7828c00640 1 --2- 172.21.15.143:0/1703576568 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f77fc06b2c0 0x7f77fc06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:00.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.611+0000 7f7828c00640 1 --2- 172.21.15.143:0/1703576568 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f77fc06b2c0 0x7f77fc06d780 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7814006fd0 tx=0x7f7814008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:00.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.614+0000 7f7820c00640 1 -- 172.21.15.143:0/1703576568 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7824155e70 con 0x7f782415b440 2024-01-16T15:01:00.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:00.827+0000 7f782b600640 1 -- 172.21.15.143:0/1703576568 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f78240cc3a0 con 0x7f782415b440 2024-01-16T15:01:01.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.393+0000 7f7820c00640 1 -- 172.21.15.143:0/1703576568 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f78240cc3a0 con 0x7f782415b440 2024-01-16T15:01:01.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.396+0000 7f782b600640 1 -- 172.21.15.143:0/1703576568 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f77fc06b2c0 msgr2=0x7f77fc06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:01.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.396+0000 7f782b600640 1 --2- 172.21.15.143:0/1703576568 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f77fc06b2c0 0x7f77fc06d780 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7814006fd0 tx=0x7f7814008040 comp rx=0 tx=0).stop 2024-01-16T15:01:01.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.396+0000 7f782b600640 1 -- 172.21.15.143:0/1703576568 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f782415b440 msgr2=0x7f7824159a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:01.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.396+0000 7f782b600640 1 --2- 172.21.15.143:0/1703576568 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f782415b440 0x7f7824159a80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f781c00c7d0 tx=0x7f781c00cc90 comp rx=0 tx=0).stop 2024-01-16T15:01:01.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.396+0000 7f782b600640 1 -- 172.21.15.143:0/1703576568 shutdown_connections 2024-01-16T15:01:01.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.396+0000 7f782b600640 1 --2- 172.21.15.143:0/1703576568 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f77fc06b2c0 0x7f77fc06d780 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:01.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.396+0000 7f782b600640 1 --2- 172.21.15.143:0/1703576568 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f782415b440 0x7f7824159a80 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:01.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.396+0000 7f782b600640 1 --2- 172.21.15.143:0/1703576568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78240cbf20 0x7f782415b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:01.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.396+0000 7f782b600640 1 --2- 172.21.15.143:0/1703576568 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78240c57e0 0x7f782415e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:01.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.396+0000 7f782b600640 1 -- 172.21.15.143:0/1703576568 >> 172.21.15.143:0/1703576568 conn(0x7f78240b0c50 msgr2=0x7f7824070c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:01.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.397+0000 7f782b600640 1 -- 172.21.15.143:0/1703576568 shutdown_connections 2024-01-16T15:01:01.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.397+0000 7f782b600640 1 -- 172.21.15.143:0/1703576568 wait complete. 2024-01-16T15:01:01.399 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-16T15:01:01.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.625+0000 7f6e9ac00640 1 Processor -- start 2024-01-16T15:01:01.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.625+0000 7f6e9ac00640 1 -- start start 2024-01-16T15:01:01.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.627+0000 7f6e9ac00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6e94143be0 0x7f6e9415e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:01.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.627+0000 7f6e9ac00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e940b3f40 con 0x7f6e94143800 2024-01-16T15:01:01.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.627+0000 7f6e9ac00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e940b3c60 con 0x7f6e9415b460 2024-01-16T15:01:01.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.627+0000 7f6e9ac00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e9415dde0 con 0x7f6e94143be0 2024-01-16T15:01:01.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.627+0000 7f6e99800640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f6e94143800 0x7f6e940b3380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43426/0 (socket says 172.21.15.143:43426) 2024-01-16T15:01:01.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.627+0000 7f6e99800640 1 -- 172.21.15.143:0/3031668826 learned_addr learned my addr 172.21.15.143:0/3031668826 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:01:01.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.627+0000 7f6e98e00640 1 --2- 172.21.15.143:0/3031668826 >> v2:172.21.15.143:3300/0 conn(0x7f6e94143be0 0x7f6e9415e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:01.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.628+0000 7f6e93e00640 1 -- 172.21.15.143:0/3031668826 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3516286219 0 0) 0x7f6e940b3f40 con 0x7f6e94143800 2024-01-16T15:01:01.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.628+0000 7f6e93e00640 1 -- 172.21.15.143:0/3031668826 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e74005730 con 0x7f6e94143800 2024-01-16T15:01:01.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.628+0000 7f6e98e00640 1 -- 172.21.15.143:0/3031668826 >> v1:172.21.15.204:6789/0 conn(0x7f6e9415b460 legacy=0x7f6e940cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:01.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.628+0000 7f6e98e00640 1 -- 172.21.15.143:0/3031668826 >> v1:172.21.15.143:6789/0 conn(0x7f6e94143800 legacy=0x7f6e940b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:01.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.628+0000 7f6e98e00640 1 -- 172.21.15.143:0/3031668826 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e9415e0c0 con 0x7f6e94143be0 2024-01-16T15:01:01.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.629+0000 7f6e98e00640 1 --2- 172.21.15.143:0/3031668826 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6e94143be0 0x7f6e9415e8c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c0097a0 tx=0x7f6e7c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8473c29b060de536 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:01.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.629+0000 7f6e93e00640 1 -- 172.21.15.143:0/3031668826 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c031830 con 0x7f6e94143be0 2024-01-16T15:01:01.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.629+0000 7f6e93e00640 1 -- 172.21.15.143:0/3031668826 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e7c031990 con 0x7f6e94143be0 2024-01-16T15:01:01.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.629+0000 7f6e93e00640 1 -- 172.21.15.143:0/3031668826 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c031d20 con 0x7f6e94143be0 2024-01-16T15:01:01.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.630+0000 7f6e9ac00640 1 -- 172.21.15.143:0/3031668826 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6e94143be0 msgr2=0x7f6e9415e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:01.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.630+0000 7f6e9ac00640 1 --2- 172.21.15.143:0/3031668826 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6e94143be0 0x7f6e9415e8c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c0097a0 tx=0x7f6e7c02b8d0 comp rx=0 tx=0).stop 2024-01-16T15:01:01.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.630+0000 7f6e9ac00640 1 -- 172.21.15.143:0/3031668826 shutdown_connections 2024-01-16T15:01:01.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.630+0000 7f6e9ac00640 1 --2- 172.21.15.143:0/3031668826 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6e94143be0 0x7f6e9415e8c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:01.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.630+0000 7f6e9ac00640 1 -- 172.21.15.143:0/3031668826 >> 172.21.15.143:0/3031668826 conn(0x7f6e9406dc30 msgr2=0x7f6e9406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:01.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.630+0000 7f6e9ac00640 1 -- 172.21.15.143:0/3031668826 shutdown_connections 2024-01-16T15:01:01.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.630+0000 7f6e9ac00640 1 -- 172.21.15.143:0/3031668826 wait complete. 2024-01-16T15:01:01.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.632+0000 7f6e9ac00640 1 Processor -- start 2024-01-16T15:01:01.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.632+0000 7f6e9ac00640 1 -- start start 2024-01-16T15:01:01.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.633+0000 7f6e9ac00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6e94143800 0x7f6e9415d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:01.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.633+0000 7f6e9ac00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6e94143be0 0x7f6e940aaed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:01.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.633+0000 7f6e9ac00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e9415b460 0x7f6e94159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:01.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.633+0000 7f6e9ac00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e940cae20 con 0x7f6e94143800 2024-01-16T15:01:01.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.633+0000 7f6e9ac00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e940cb350 con 0x7f6e9415b460 2024-01-16T15:01:01.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.633+0000 7f6e9ac00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e940cb070 con 0x7f6e94143be0 2024-01-16T15:01:01.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.633+0000 7f6e99800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6e94143800 0x7f6e9415d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:01.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.633+0000 7f6e99800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6e94143800 0x7f6e9415d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48208/0 (socket says 172.21.15.143:48208) 2024-01-16T15:01:01.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.633+0000 7f6e99800640 1 -- 172.21.15.143:0/170251237 learned_addr learned my addr 172.21.15.143:0/170251237 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:01.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.633+0000 7f6e9a200640 1 --2- 172.21.15.143:0/170251237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e9415b460 0x7f6e94159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:01.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.633+0000 7f6e98e00640 1 --2- 172.21.15.143:0/170251237 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6e94143be0 0x7f6e940aaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:01.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.633+0000 7f6e99800640 1 -- 172.21.15.143:0/170251237 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6e94143be0 msgr2=0x7f6e940aaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:01.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.633+0000 7f6e99800640 1 --2- 172.21.15.143:0/170251237 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6e94143be0 0x7f6e940aaed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:01.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.633+0000 7f6e99800640 1 -- 172.21.15.143:0/170251237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e9415b460 msgr2=0x7f6e94159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:01.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.633+0000 7f6e99800640 1 --2- 172.21.15.143:0/170251237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e9415b460 0x7f6e94159790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:01.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.633+0000 7f6e99800640 1 -- 172.21.15.143:0/170251237 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7c009400 con 0x7f6e94143800 2024-01-16T15:01:01.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.634+0000 7f6e99800640 1 --2- 172.21.15.143:0/170251237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6e94143800 0x7f6e9415d010 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6e8400c500 tx=0x7f6e8400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:01.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.634+0000 7f6e91600640 1 -- 172.21.15.143:0/170251237 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e84005ce0 con 0x7f6e94143800 2024-01-16T15:01:01.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.634+0000 7f6e91600640 1 -- 172.21.15.143:0/170251237 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e84005e40 con 0x7f6e94143800 2024-01-16T15:01:01.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.634+0000 7f6e9ac00640 1 -- 172.21.15.143:0/170251237 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e940a84d0 con 0x7f6e94143800 2024-01-16T15:01:01.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.634+0000 7f6e91600640 1 -- 172.21.15.143:0/170251237 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e84007330 con 0x7f6e94143800 2024-01-16T15:01:01.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.634+0000 7f6e9ac00640 1 -- 172.21.15.143:0/170251237 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e940a8a90 con 0x7f6e94143800 2024-01-16T15:01:01.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.635+0000 7f6e9ac00640 1 -- 172.21.15.143:0/170251237 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e940a8c80 con 0x7f6e94143800 2024-01-16T15:01:01.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.636+0000 7f6e91600640 1 -- 172.21.15.143:0/170251237 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f6e84007550 con 0x7f6e94143800 2024-01-16T15:01:01.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.638+0000 7f6e91600640 1 --2- 172.21.15.143:0/170251237 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6e6c06b2c0 0x7f6e6c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:01.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.638+0000 7f6e91600640 1 -- 172.21.15.143:0/170251237 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f6e84089350 con 0x7f6e94143800 2024-01-16T15:01:01.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.638+0000 7f6e98e00640 1 --2- 172.21.15.143:0/170251237 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6e6c06b2c0 0x7f6e6c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:01.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.638+0000 7f6e98e00640 1 --2- 172.21.15.143:0/170251237 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6e6c06b2c0 0x7f6e6c06d780 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c036000 tx=0x7f6e7c008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:01.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.642+0000 7f6e91600640 1 -- 172.21.15.143:0/170251237 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6e840541a0 con 0x7f6e94143800 2024-01-16T15:01:01.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:01.819+0000 7f6e9ac00640 1 -- 172.21.15.143:0/170251237 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6e9404e840 con 0x7f6e94143800 2024-01-16T15:01:02.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:02.793+0000 7f6e91600640 1 -- 172.21.15.143:0/170251237 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7f6e840577c0 con 0x7f6e94143800 2024-01-16T15:01:02.775 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' removed 2024-01-16T15:01:02.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:02.798+0000 7f6e9ac00640 1 -- 172.21.15.143:0/170251237 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6e6c06b2c0 msgr2=0x7f6e6c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:02.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:02.798+0000 7f6e9ac00640 1 --2- 172.21.15.143:0/170251237 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6e6c06b2c0 0x7f6e6c06d780 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c036000 tx=0x7f6e7c008030 comp rx=0 tx=0).stop 2024-01-16T15:01:02.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:02.798+0000 7f6e9ac00640 1 -- 172.21.15.143:0/170251237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6e94143800 msgr2=0x7f6e9415d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:02.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:02.798+0000 7f6e9ac00640 1 --2- 172.21.15.143:0/170251237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6e94143800 0x7f6e9415d010 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6e8400c500 tx=0x7f6e8400c9c0 comp rx=0 tx=0).stop 2024-01-16T15:01:02.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:02.799+0000 7f6e9ac00640 1 -- 172.21.15.143:0/170251237 shutdown_connections 2024-01-16T15:01:02.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:02.799+0000 7f6e9ac00640 1 --2- 172.21.15.143:0/170251237 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f6e6c06b2c0 0x7f6e6c06d780 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:02.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:02.799+0000 7f6e9ac00640 1 --2- 172.21.15.143:0/170251237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e9415b460 0x7f6e94159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:02.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:02.799+0000 7f6e9ac00640 1 --2- 172.21.15.143:0/170251237 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6e94143be0 0x7f6e940aaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:02.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:02.799+0000 7f6e9ac00640 1 --2- 172.21.15.143:0/170251237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6e94143800 0x7f6e9415d010 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:02.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:02.799+0000 7f6e9ac00640 1 -- 172.21.15.143:0/170251237 >> 172.21.15.143:0/170251237 conn(0x7f6e9406dc30 msgr2=0x7f6e9406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:02.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:02.799+0000 7f6e9ac00640 1 -- 172.21.15.143:0/170251237 shutdown_connections 2024-01-16T15:01:02.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:02.799+0000 7f6e9ac00640 1 -- 172.21.15.143:0/170251237 wait complete. 2024-01-16T15:01:02.801 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-16T15:01:03.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.034+0000 7fd20a600640 1 Processor -- start 2024-01-16T15:01:03.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.034+0000 7fd20a600640 1 -- start start 2024-01-16T15:01:03.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.036+0000 7fd20a600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd204144510 0x7fd20406d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:03.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.036+0000 7fd20a600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd20415edd0 0x7fd2040b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:03.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.036+0000 7fd20a600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2041448f0 con 0x7fd2040ccd00 2024-01-16T15:01:03.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.036+0000 7fd20a600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd204144d80 con 0x7fd204144510 2024-01-16T15:01:03.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.036+0000 7fd20a600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd20415f2d0 con 0x7fd20415edd0 2024-01-16T15:01:03.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.036+0000 7fd209200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd204144510 0x7fd20406d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:03.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.036+0000 7fd209c00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd2040ccd00 0x7fd2040b4180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43432/0 (socket says 172.21.15.143:43432) 2024-01-16T15:01:03.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.036+0000 7fd209c00640 1 -- 172.21.15.143:0/2602188527 learned_addr learned my addr 172.21.15.143:0/2602188527 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:01:03.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.037+0000 7fd203e00640 1 --2- 172.21.15.143:0/2602188527 >> v2:172.21.15.204:3300/0 conn(0x7fd20415edd0 0x7fd2040b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:03.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.037+0000 7fd209200640 1 -- 172.21.15.143:0/2602188527 >> v2:172.21.15.204:3300/0 conn(0x7fd20415edd0 msgr2=0x7fd2040b0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:03.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.037+0000 7fd209200640 1 --2- 172.21.15.143:0/2602188527 >> v2:172.21.15.204:3300/0 conn(0x7fd20415edd0 0x7fd2040b0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:03.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.037+0000 7fd209200640 1 -- 172.21.15.143:0/2602188527 >> v1:172.21.15.143:6789/0 conn(0x7fd2040ccd00 legacy=0x7fd2040b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:03.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.037+0000 7fd209200640 1 -- 172.21.15.143:0/2602188527 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd20415f5b0 con 0x7fd204144510 2024-01-16T15:01:03.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.037+0000 7fd203400640 1 -- 172.21.15.143:0/2602188527 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1870202332 0 0) 0x7fd2041448f0 con 0x7fd2040ccd00 2024-01-16T15:01:03.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.038+0000 7fd209200640 1 --2- 172.21.15.143:0/2602188527 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd204144510 0x7fd20406d7b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd2040b3950 tx=0x7fd1fc02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3bf03d9b433772d server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:03.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.038+0000 7fd203400640 1 -- 172.21.15.143:0/2602188527 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc02cbf0 con 0x7fd204144510 2024-01-16T15:01:03.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.038+0000 7fd203400640 1 -- 172.21.15.143:0/2602188527 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1fc02cd50 con 0x7fd204144510 2024-01-16T15:01:03.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.038+0000 7fd203400640 1 -- 172.21.15.143:0/2602188527 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc034660 con 0x7fd204144510 2024-01-16T15:01:03.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.039+0000 7fd20a600640 1 -- 172.21.15.143:0/2602188527 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd204144510 msgr2=0x7fd20406d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:03.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.039+0000 7fd20a600640 1 --2- 172.21.15.143:0/2602188527 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd204144510 0x7fd20406d7b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd2040b3950 tx=0x7fd1fc02be40 comp rx=0 tx=0).stop 2024-01-16T15:01:03.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.039+0000 7fd20a600640 1 -- 172.21.15.143:0/2602188527 shutdown_connections 2024-01-16T15:01:03.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.039+0000 7fd20a600640 1 --2- 172.21.15.143:0/2602188527 >> v2:172.21.15.204:3300/0 conn(0x7fd20415edd0 0x7fd2040b0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:03.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.039+0000 7fd20a600640 1 --2- 172.21.15.143:0/2602188527 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd204144510 0x7fd20406d7b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:03.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.039+0000 7fd20a600640 1 -- 172.21.15.143:0/2602188527 >> 172.21.15.143:0/2602188527 conn(0x7fd2040cb8a0 msgr2=0x7fd2040cbc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:03.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.039+0000 7fd20a600640 1 -- 172.21.15.143:0/2602188527 shutdown_connections 2024-01-16T15:01:03.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.039+0000 7fd20a600640 1 -- 172.21.15.143:0/2602188527 wait complete. 2024-01-16T15:01:03.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.041+0000 7fd20a600640 1 Processor -- start 2024-01-16T15:01:03.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.041+0000 7fd20a600640 1 -- start start 2024-01-16T15:01:03.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.042+0000 7fd20a600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2040ccd00 0x7fd2040702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:03.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.042+0000 7fd20a600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd204144510 0x7fd2041f22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:03.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.042+0000 7fd20a600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd20415edd0 0x7fd20413f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:03.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.042+0000 7fd20a600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd204141e50 con 0x7fd204144510 2024-01-16T15:01:03.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.042+0000 7fd20a600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd204141310 con 0x7fd2040ccd00 2024-01-16T15:01:03.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.042+0000 7fd20a600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2041415b0 con 0x7fd20415edd0 2024-01-16T15:01:03.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.042+0000 7fd203e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd204144510 0x7fd2041f22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:03.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.042+0000 7fd203e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd204144510 0x7fd2041f22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48236/0 (socket says 172.21.15.143:48236) 2024-01-16T15:01:03.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.042+0000 7fd203e00640 1 -- 172.21.15.143:0/95338354 learned_addr learned my addr 172.21.15.143:0/95338354 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:03.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.042+0000 7fd203e00640 1 -- 172.21.15.143:0/95338354 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd20415edd0 msgr2=0x7fd20413f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:03.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.042+0000 7fd203e00640 1 --2- 172.21.15.143:0/95338354 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd20415edd0 0x7fd20413f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:03.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.042+0000 7fd209200640 1 --2- 172.21.15.143:0/95338354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2040ccd00 0x7fd2040702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:03.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.042+0000 7fd203e00640 1 -- 172.21.15.143:0/95338354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2040ccd00 msgr2=0x7fd2040702f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:03.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.042+0000 7fd203e00640 1 --2- 172.21.15.143:0/95338354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2040ccd00 0x7fd2040702f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:03.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.042+0000 7fd203e00640 1 -- 172.21.15.143:0/95338354 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1fc009a70 con 0x7fd204144510 2024-01-16T15:01:03.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.043+0000 7fd203e00640 1 --2- 172.21.15.143:0/95338354 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd204144510 0x7fd2041f22f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00d800 tx=0x7fd1ec00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:03.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.043+0000 7fd200c00640 1 -- 172.21.15.143:0/95338354 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec018070 con 0x7fd204144510 2024-01-16T15:01:03.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.043+0000 7fd200c00640 1 -- 172.21.15.143:0/95338354 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1ec005990 con 0x7fd204144510 2024-01-16T15:01:03.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.043+0000 7fd200c00640 1 -- 172.21.15.143:0/95338354 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec005b20 con 0x7fd204144510 2024-01-16T15:01:03.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.043+0000 7fd20a600640 1 -- 172.21.15.143:0/95338354 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd204141890 con 0x7fd204144510 2024-01-16T15:01:03.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.043+0000 7fd20a600640 1 -- 172.21.15.143:0/95338354 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd204070830 con 0x7fd204144510 2024-01-16T15:01:03.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.044+0000 7fd20a600640 1 -- 172.21.15.143:0/95338354 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd20404e840 con 0x7fd204144510 2024-01-16T15:01:03.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.045+0000 7fd200c00640 1 -- 172.21.15.143:0/95338354 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fd1ec005c80 con 0x7fd204144510 2024-01-16T15:01:03.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.046+0000 7fd200c00640 1 --2- 172.21.15.143:0/95338354 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd1dc06b2c0 0x7fd1dc06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:03.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.046+0000 7fd200c00640 1 -- 172.21.15.143:0/95338354 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5473+0+0 (secure 0 0 0) 0x7fd1ec089880 con 0x7fd204144510 2024-01-16T15:01:03.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.047+0000 7fd209200640 1 --2- 172.21.15.143:0/95338354 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd1dc06b2c0 0x7fd1dc06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:03.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.047+0000 7fd209200640 1 --2- 172.21.15.143:0/95338354 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd1dc06b2c0 0x7fd1dc06d780 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc036000 tx=0x7fd1fc030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:03.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.051+0000 7fd200c00640 1 -- 172.21.15.143:0/95338354 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd1ec054860 con 0x7fd204144510 2024-01-16T15:01:03.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.240+0000 7fd20a600640 1 -- 172.21.15.143:0/95338354 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd20406c9c0 con 0x7fd204144510 2024-01-16T15:01:03.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.796+0000 7fd200c00640 1 -- 172.21.15.143:0/95338354 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd1ec057e80 con 0x7fd204144510 2024-01-16T15:01:03.778 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' removed 2024-01-16T15:01:03.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.801+0000 7fd20a600640 1 -- 172.21.15.143:0/95338354 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd1dc06b2c0 msgr2=0x7fd1dc06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:03.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.801+0000 7fd20a600640 1 --2- 172.21.15.143:0/95338354 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd1dc06b2c0 0x7fd1dc06d780 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc036000 tx=0x7fd1fc030000 comp rx=0 tx=0).stop 2024-01-16T15:01:03.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.801+0000 7fd20a600640 1 -- 172.21.15.143:0/95338354 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd204144510 msgr2=0x7fd2041f22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:03.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.801+0000 7fd20a600640 1 --2- 172.21.15.143:0/95338354 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd204144510 0x7fd2041f22f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00d800 tx=0x7fd1ec00dcc0 comp rx=0 tx=0).stop 2024-01-16T15:01:03.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.801+0000 7fd20a600640 1 -- 172.21.15.143:0/95338354 shutdown_connections 2024-01-16T15:01:03.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.801+0000 7fd20a600640 1 --2- 172.21.15.143:0/95338354 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd1dc06b2c0 0x7fd1dc06d780 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:03.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.801+0000 7fd20a600640 1 --2- 172.21.15.143:0/95338354 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd20415edd0 0x7fd20413f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:03.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.801+0000 7fd20a600640 1 --2- 172.21.15.143:0/95338354 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd204144510 0x7fd2041f22f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:03.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.801+0000 7fd20a600640 1 --2- 172.21.15.143:0/95338354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2040ccd00 0x7fd2040702f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:03.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.801+0000 7fd20a600640 1 -- 172.21.15.143:0/95338354 >> 172.21.15.143:0/95338354 conn(0x7fd2040cb8a0 msgr2=0x7fd2040af230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:03.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.801+0000 7fd20a600640 1 -- 172.21.15.143:0/95338354 shutdown_connections 2024-01-16T15:01:03.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:03.802+0000 7fd20a600640 1 -- 172.21.15.143:0/95338354 wait complete. 2024-01-16T15:01:03.814 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-16T15:01:03.814 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-16T15:01:03.814 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-16T15:01:03.815 INFO:teuthology.orchestra.run:waiting for 300 2024-01-16T15:01:03.815 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-16T15:01:03.815 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-16T15:01:03.815 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-16T15:01:03.815 INFO:teuthology.orchestra.run:waiting for 300 2024-01-16T15:01:03.815 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-16T15:01:03.815 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-16T15:01:03.816 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-16T15:01:03.816 INFO:teuthology.orchestra.run:waiting for 300 2024-01-16T15:01:03.816 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-16T15:01:03.816 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-16T15:01:03.816 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-16T15:01:03.816 INFO:teuthology.orchestra.run:waiting for 300 2024-01-16T15:01:03.837 INFO:tasks.ceph.mds.d:Stopped 2024-01-16T15:01:03.838 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-16T15:01:03.840 INFO:tasks.ceph.mds.b:Stopped 2024-01-16T15:01:03.840 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-16T15:01:03.840 INFO:tasks.ceph.mds.c:Stopped 2024-01-16T15:01:03.840 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-16T15:01:03.883 INFO:tasks.ceph.mds.d:Started 2024-01-16T15:01:03.884 DEBUG:teuthology.parallel:result is None 2024-01-16T15:01:03.886 INFO:tasks.ceph.mds.c:Started 2024-01-16T15:01:03.886 DEBUG:teuthology.parallel:result is None 2024-01-16T15:01:03.887 INFO:tasks.ceph.mds.b:Started 2024-01-16T15:01:03.887 DEBUG:teuthology.parallel:result is None 2024-01-16T15:01:04.034 INFO:tasks.ceph.mds.b.smithi204.stdout:starting mds.b at 2024-01-16T15:01:04.036 INFO:tasks.ceph.mds.c.smithi143.stdout:starting mds.c at 2024-01-16T15:01:04.036 INFO:tasks.ceph.mds.d.smithi204.stdout:starting mds.d at 2024-01-16T15:01:09.839 INFO:tasks.ceph.mds.a:Stopped 2024-01-16T15:01:09.839 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-16T15:01:09.847 INFO:tasks.ceph.mds.a:Started 2024-01-16T15:01:09.848 DEBUG:teuthology.parallel:result is None 2024-01-16T15:01:09.848 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-16T15:01:10.002 INFO:tasks.ceph.mds.a.smithi143.stdout:starting mds.a at 2024-01-16T15:01:10.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.054+0000 7fb454800640 1 Processor -- start 2024-01-16T15:01:10.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.054+0000 7fb454800640 1 -- start start 2024-01-16T15:01:10.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.055+0000 7fb454800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb44c0af000 0x7fb44c0b3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:10.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.055+0000 7fb454800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb44c06dca0 con 0x7fb44c161060 2024-01-16T15:01:10.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.055+0000 7fb454800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb44c06df40 con 0x7fb44c06b680 2024-01-16T15:01:10.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.055+0000 7fb454800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb44c0af3e0 con 0x7fb44c0af000 2024-01-16T15:01:10.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.056+0000 7fb451e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb44c0af000 0x7fb44c0b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:10.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.056+0000 7fb451e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb44c0af000 0x7fb44c0b3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40282/0 (socket says 172.21.15.143:40282) 2024-01-16T15:01:10.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.056+0000 7fb451e00640 1 -- 172.21.15.143:0/1093088906 learned_addr learned my addr 172.21.15.143:0/1093088906 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:10.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.056+0000 7fb451e00640 1 -- 172.21.15.143:0/1093088906 >> v1:172.21.15.204:6789/0 conn(0x7fb44c06b680 legacy=0x7fb44c15e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:10.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.056+0000 7fb451e00640 1 -- 172.21.15.143:0/1093088906 >> v1:172.21.15.143:6789/0 conn(0x7fb44c161060 legacy=0x7fb44c15da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:10.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.056+0000 7fb450a00640 1 -- 172.21.15.143:0/1093088906 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3134641162 0 0) 0x7fb44c06dca0 con 0x7fb44c161060 2024-01-16T15:01:10.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.056+0000 7fb451e00640 1 -- 172.21.15.143:0/1093088906 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb44c06e220 con 0x7fb44c0af000 2024-01-16T15:01:10.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.057+0000 7fb451e00640 1 --2- 172.21.15.143:0/1093088906 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb44c0af000 0x7fb44c0b3330 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb43c0098a0 tx=0x7fb43c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b6f4d0242c927f server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:10.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.058+0000 7fb450a00640 1 -- 172.21.15.143:0/1093088906 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43c030840 con 0x7fb44c0af000 2024-01-16T15:01:10.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.058+0000 7fb450a00640 1 -- 172.21.15.143:0/1093088906 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb43c0309a0 con 0x7fb44c0af000 2024-01-16T15:01:10.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.058+0000 7fb450a00640 1 -- 172.21.15.143:0/1093088906 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43c030d30 con 0x7fb44c0af000 2024-01-16T15:01:10.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.058+0000 7fb454800640 1 -- 172.21.15.143:0/1093088906 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb44c0af000 msgr2=0x7fb44c0b3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:10.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.058+0000 7fb454800640 1 --2- 172.21.15.143:0/1093088906 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb44c0af000 0x7fb44c0b3330 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb43c0098a0 tx=0x7fb43c02b980 comp rx=0 tx=0).stop 2024-01-16T15:01:10.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.058+0000 7fb454800640 1 -- 172.21.15.143:0/1093088906 shutdown_connections 2024-01-16T15:01:10.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.058+0000 7fb454800640 1 --2- 172.21.15.143:0/1093088906 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb44c0af000 0x7fb44c0b3330 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:10.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.058+0000 7fb454800640 1 -- 172.21.15.143:0/1093088906 >> 172.21.15.143:0/1093088906 conn(0x7fb44c0cf900 msgr2=0x7fb44c0cfd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:10.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.058+0000 7fb454800640 1 -- 172.21.15.143:0/1093088906 shutdown_connections 2024-01-16T15:01:10.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.058+0000 7fb454800640 1 -- 172.21.15.143:0/1093088906 wait complete. 2024-01-16T15:01:10.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.060+0000 7fb454800640 1 Processor -- start 2024-01-16T15:01:10.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.060+0000 7fb454800640 1 -- start start 2024-01-16T15:01:10.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.061+0000 7fb454800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb44c06b680 0x7fb44c13fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:10.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.061+0000 7fb454800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb44c0af000 0x7fb44c1580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:10.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.061+0000 7fb454800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb44c161060 0x7fb44c152800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:10.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.061+0000 7fb454800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c141d30 con 0x7fb44c161060 2024-01-16T15:01:10.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.061+0000 7fb454800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c1410b0 con 0x7fb44c0af000 2024-01-16T15:01:10.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.061+0000 7fb454800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c141390 con 0x7fb44c06b680 2024-01-16T15:01:10.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.061+0000 7fb452800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb44c161060 0x7fb44c152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:10.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.061+0000 7fb452800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb44c161060 0x7fb44c152800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40292/0 (socket says 172.21.15.143:40292) 2024-01-16T15:01:10.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.061+0000 7fb452800640 1 -- 172.21.15.143:0/3686604798 learned_addr learned my addr 172.21.15.143:0/3686604798 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:10.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.062+0000 7fb451e00640 1 --2- 172.21.15.143:0/3686604798 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb44c06b680 0x7fb44c13fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:10.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.062+0000 7fb451400640 1 --2- 172.21.15.143:0/3686604798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb44c0af000 0x7fb44c1580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:10.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.062+0000 7fb452800640 1 -- 172.21.15.143:0/3686604798 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb44c06b680 msgr2=0x7fb44c13fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:10.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.062+0000 7fb452800640 1 --2- 172.21.15.143:0/3686604798 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb44c06b680 0x7fb44c13fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:10.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.062+0000 7fb452800640 1 -- 172.21.15.143:0/3686604798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb44c0af000 msgr2=0x7fb44c1580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:10.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.062+0000 7fb452800640 1 --2- 172.21.15.143:0/3686604798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb44c0af000 0x7fb44c1580a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:10.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.062+0000 7fb452800640 1 -- 172.21.15.143:0/3686604798 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb43c0094d0 con 0x7fb44c161060 2024-01-16T15:01:10.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.062+0000 7fb451e00640 1 --2- 172.21.15.143:0/3686604798 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb44c06b680 0x7fb44c13fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:01:10.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.062+0000 7fb451400640 1 --2- 172.21.15.143:0/3686604798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb44c0af000 0x7fb44c1580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:01:10.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.062+0000 7fb452800640 1 --2- 172.21.15.143:0/3686604798 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb44c161060 0x7fb44c152800 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb44400c780 tx=0x7fb44400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:10.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.062+0000 7fb44a000640 1 -- 172.21.15.143:0/3686604798 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb444003d80 con 0x7fb44c161060 2024-01-16T15:01:10.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.062+0000 7fb44a000640 1 -- 172.21.15.143:0/3686604798 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb44400ce30 con 0x7fb44c161060 2024-01-16T15:01:10.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.062+0000 7fb454800640 1 -- 172.21.15.143:0/3686604798 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb44c141670 con 0x7fb44c161060 2024-01-16T15:01:10.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.062+0000 7fb44a000640 1 -- 172.21.15.143:0/3686604798 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb444005290 con 0x7fb44c161060 2024-01-16T15:01:10.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.063+0000 7fb454800640 1 -- 172.21.15.143:0/3686604798 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb44c1401f0 con 0x7fb44c161060 2024-01-16T15:01:10.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.064+0000 7fb454800640 1 -- 172.21.15.143:0/3686604798 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb44c141e70 con 0x7fb44c161060 2024-01-16T15:01:10.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.065+0000 7fb44a000640 1 -- 172.21.15.143:0/3686604798 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fb4440053f0 con 0x7fb44c161060 2024-01-16T15:01:10.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.065+0000 7fb44a000640 1 --2- 172.21.15.143:0/3686604798 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb42406b2f0 0x7fb42406d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:10.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.066+0000 7fb44a000640 1 -- 172.21.15.143:0/3686604798 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb444088dc0 con 0x7fb44c161060 2024-01-16T15:01:10.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.066+0000 7fb451e00640 1 --2- 172.21.15.143:0/3686604798 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb42406b2f0 0x7fb42406d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:10.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.067+0000 7fb451e00640 1 --2- 172.21.15.143:0/3686604798 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb42406b2f0 0x7fb42406d7b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb43c036000 tx=0x7fb43c03b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:10.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.071+0000 7fb44a000640 1 -- 172.21.15.143:0/3686604798 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb44c141e70 con 0x7fb44c161060 2024-01-16T15:01:10.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:10.274+0000 7fb454800640 1 -- 172.21.15.143:0/3686604798 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fb44c141e70 con 0x7fb44c161060 2024-01-16T15:01:11.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:11.791+0000 7fb44a000640 1 -- 172.21.15.143:0/3686604798 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v30) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb44c141e70 con 0x7fb44c161060 2024-01-16T15:01:11.772 INFO:teuthology.orchestra.run.smithi143.stderr: removed all blocklist entries 2024-01-16T15:01:11.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:11.795+0000 7fb454800640 1 -- 172.21.15.143:0/3686604798 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb42406b2f0 msgr2=0x7fb42406d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:11.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:11.795+0000 7fb454800640 1 --2- 172.21.15.143:0/3686604798 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb42406b2f0 0x7fb42406d7b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb43c036000 tx=0x7fb43c03b040 comp rx=0 tx=0).stop 2024-01-16T15:01:11.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:11.795+0000 7fb454800640 1 -- 172.21.15.143:0/3686604798 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb44c161060 msgr2=0x7fb44c152800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:11.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:11.795+0000 7fb454800640 1 --2- 172.21.15.143:0/3686604798 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb44c161060 0x7fb44c152800 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb44400c780 tx=0x7fb44400cc40 comp rx=0 tx=0).stop 2024-01-16T15:01:11.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:11.796+0000 7fb454800640 1 -- 172.21.15.143:0/3686604798 shutdown_connections 2024-01-16T15:01:11.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:11.796+0000 7fb454800640 1 --2- 172.21.15.143:0/3686604798 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb42406b2f0 0x7fb42406d7b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:11.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:11.796+0000 7fb454800640 1 --2- 172.21.15.143:0/3686604798 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb44c161060 0x7fb44c152800 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:11.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:11.796+0000 7fb454800640 1 --2- 172.21.15.143:0/3686604798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb44c0af000 0x7fb44c1580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:11.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:11.796+0000 7fb454800640 1 --2- 172.21.15.143:0/3686604798 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb44c06b680 0x7fb44c13fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:11.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:11.796+0000 7fb454800640 1 -- 172.21.15.143:0/3686604798 >> 172.21.15.143:0/3686604798 conn(0x7fb44c0cf900 msgr2=0x7fb44c0cfce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:11.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:11.796+0000 7fb454800640 1 -- 172.21.15.143:0/3686604798 shutdown_connections 2024-01-16T15:01:11.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:11.796+0000 7fb454800640 1 -- 172.21.15.143:0/3686604798 wait complete. 2024-01-16T15:01:11.798 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-16T15:01:12.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.020+0000 7f7882400640 1 Processor -- start 2024-01-16T15:01:12.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.020+0000 7f7882400640 1 -- start start 2024-01-16T15:01:12.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.021+0000 7f7882400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f787c0cdfb0 0x7f787c0ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:12.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.021+0000 7f7882400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f787c0b20c0 0x7f787c0b2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:12.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.021+0000 7f7882400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f787c0ce950 con 0x7f787c15a4f0 2024-01-16T15:01:12.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.021+0000 7f7882400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f787c0b3a40 con 0x7f787c0b20c0 2024-01-16T15:01:12.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.021+0000 7f7882400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f787c0b2e30 con 0x7f787c0cdfb0 2024-01-16T15:01:12.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.022+0000 7f787be00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f787c0b20c0 0x7f787c0b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:12.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.022+0000 7f787be00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f787c0b20c0 0x7f787c0b2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40308/0 (socket says 172.21.15.143:40308) 2024-01-16T15:01:12.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.022+0000 7f787be00640 1 -- 172.21.15.143:0/2016454407 learned_addr learned my addr 172.21.15.143:0/2016454407 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:12.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.022+0000 7f7872800640 1 --2- 172.21.15.143:0/2016454407 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f787c0cdfb0 0x7f787c0ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:12.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.022+0000 7f787be00640 1 -- 172.21.15.143:0/2016454407 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f787c0cdfb0 msgr2=0x7f787c0ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:12.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.022+0000 7f787be00640 1 --2- 172.21.15.143:0/2016454407 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f787c0cdfb0 0x7f787c0ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:12.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.022+0000 7f787be00640 1 -- 172.21.15.143:0/2016454407 >> v1:172.21.15.204:6789/0 conn(0x7f787c15a4f0 legacy=0x7f787c0b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:12.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.022+0000 7f787be00640 1 -- 172.21.15.143:0/2016454407 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787c0b3110 con 0x7f787c0b20c0 2024-01-16T15:01:12.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.023+0000 7f787be00640 1 --2- 172.21.15.143:0/2016454407 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f787c0b20c0 0x7f787c0b2550 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7874009870 tx=0x7f787402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ef3f26067b674e server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:12.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.023+0000 7f787aa00640 1 -- 172.21.15.143:0/2016454407 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7874031640 con 0x7f787c0b20c0 2024-01-16T15:01:12.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.023+0000 7f787aa00640 1 -- 172.21.15.143:0/2016454407 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78740317a0 con 0x7f787c0b20c0 2024-01-16T15:01:12.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.023+0000 7f787aa00640 1 -- 172.21.15.143:0/2016454407 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7874031b30 con 0x7f787c0b20c0 2024-01-16T15:01:12.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.024+0000 7f7882400640 1 -- 172.21.15.143:0/2016454407 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f787c0b20c0 msgr2=0x7f787c0b2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:12.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.024+0000 7f7882400640 1 --2- 172.21.15.143:0/2016454407 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f787c0b20c0 0x7f787c0b2550 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7874009870 tx=0x7f787402b8f0 comp rx=0 tx=0).stop 2024-01-16T15:01:12.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.024+0000 7f7882400640 1 -- 172.21.15.143:0/2016454407 shutdown_connections 2024-01-16T15:01:12.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.024+0000 7f7882400640 1 --2- 172.21.15.143:0/2016454407 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f787c0b20c0 0x7f787c0b2550 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:12.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.024+0000 7f7882400640 1 --2- 172.21.15.143:0/2016454407 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f787c0cdfb0 0x7f787c0ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:12.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.024+0000 7f7882400640 1 -- 172.21.15.143:0/2016454407 >> 172.21.15.143:0/2016454407 conn(0x7f787c0b0c70 msgr2=0x7f787c0b1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:12.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.024+0000 7f7882400640 1 -- 172.21.15.143:0/2016454407 shutdown_connections 2024-01-16T15:01:12.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.024+0000 7f7882400640 1 -- 172.21.15.143:0/2016454407 wait complete. 2024-01-16T15:01:12.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.026+0000 7f7882400640 1 Processor -- start 2024-01-16T15:01:12.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.026+0000 7f7882400640 1 -- start start 2024-01-16T15:01:12.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.027+0000 7f7882400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f787c0b20c0 0x7f787c09c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:12.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.027+0000 7f7882400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f787c0cdfb0 0x7f787c15c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:12.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.027+0000 7f7882400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f787c15a4f0 0x7f787c15c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:12.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.027+0000 7f7882400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787c1550a0 con 0x7f787c0cdfb0 2024-01-16T15:01:12.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.027+0000 7f7882400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787c154520 con 0x7f787c0b20c0 2024-01-16T15:01:12.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.027+0000 7f7882400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f787c154800 con 0x7f787c15a4f0 2024-01-16T15:01:12.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.027+0000 7f7872800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f787c0cdfb0 0x7f787c15c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:12.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.027+0000 7f7872800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f787c0cdfb0 0x7f787c15c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40322/0 (socket says 172.21.15.143:40322) 2024-01-16T15:01:12.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.027+0000 7f7872800640 1 -- 172.21.15.143:0/400896961 learned_addr learned my addr 172.21.15.143:0/400896961 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:12.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.028+0000 7f787be00640 1 --2- 172.21.15.143:0/400896961 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f787c15a4f0 0x7f787c15c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:12.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.028+0000 7f787b400640 1 --2- 172.21.15.143:0/400896961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f787c0b20c0 0x7f787c09c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:12.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.028+0000 7f7872800640 1 -- 172.21.15.143:0/400896961 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f787c15a4f0 msgr2=0x7f787c15c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:12.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.028+0000 7f7872800640 1 --2- 172.21.15.143:0/400896961 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f787c15a4f0 0x7f787c15c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:12.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.028+0000 7f7872800640 1 -- 172.21.15.143:0/400896961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f787c0b20c0 msgr2=0x7f787c09c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:12.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.028+0000 7f7872800640 1 --2- 172.21.15.143:0/400896961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f787c0b20c0 0x7f787c09c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:12.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.028+0000 7f7872800640 1 -- 172.21.15.143:0/400896961 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78740094d0 con 0x7f787c0cdfb0 2024-01-16T15:01:12.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.028+0000 7f787b400640 1 --2- 172.21.15.143:0/400896961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f787c0b20c0 0x7f787c09c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:01:12.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.028+0000 7f7872800640 1 --2- 172.21.15.143:0/400896961 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f787c0cdfb0 0x7f787c15c270 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f786400d800 tx=0x7f786400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:12.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.028+0000 7f7873e00640 1 -- 172.21.15.143:0/400896961 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786400b6c0 con 0x7f787c0cdfb0 2024-01-16T15:01:12.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.028+0000 7f7873e00640 1 -- 172.21.15.143:0/400896961 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786400b820 con 0x7f787c0cdfb0 2024-01-16T15:01:12.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.028+0000 7f7873e00640 1 -- 172.21.15.143:0/400896961 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786400bbb0 con 0x7f787c0cdfb0 2024-01-16T15:01:12.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.028+0000 7f7882400640 1 -- 172.21.15.143:0/400896961 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f787c154ae0 con 0x7f787c0cdfb0 2024-01-16T15:01:12.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.029+0000 7f7882400640 1 -- 172.21.15.143:0/400896961 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f787c09cf10 con 0x7f787c0cdfb0 2024-01-16T15:01:12.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.030+0000 7f7882400640 1 -- 172.21.15.143:0/400896961 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f787c04e8b0 con 0x7f787c0cdfb0 2024-01-16T15:01:12.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.030+0000 7f7873e00640 1 -- 172.21.15.143:0/400896961 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f786400bd10 con 0x7f787c0cdfb0 2024-01-16T15:01:12.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.031+0000 7f7873e00640 1 --2- 172.21.15.143:0/400896961 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f785406b2c0 0x7f785406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:12.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.031+0000 7f7873e00640 1 -- 172.21.15.143:0/400896961 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7864089e90 con 0x7f787c0cdfb0 2024-01-16T15:01:12.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.031+0000 7f787b400640 1 --2- 172.21.15.143:0/400896961 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f785406b2c0 0x7f785406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:12.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.032+0000 7f787b400640 1 --2- 172.21.15.143:0/400896961 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f785406b2c0 0x7f785406d780 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f786c007e90 tx=0x7f786c0073a0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:12.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.036+0000 7f7873e00640 1 -- 172.21.15.143:0/400896961 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7864054fa0 con 0x7f787c0cdfb0 2024-01-16T15:01:12.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.295+0000 7f7882400640 1 -- 172.21.15.143:0/400896961 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f787c0acc00 con 0x7f787c0cdfb0 2024-01-16T15:01:12.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.296+0000 7f7873e00640 1 -- 172.21.15.143:0/400896961 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f78640585c0 con 0x7f787c0cdfb0 2024-01-16T15:01:12.277 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:01:12.277 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-16T15:01:12.278 INFO:teuthology.orchestra.run.smithi143.stdout: "auth_dump": [ 2024-01-16T15:01:12.278 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-16T15:01:12.278 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.a", 2024-01-16T15:01:12.278 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCwmaZlvfIyMxAAwY3gj2HmVRU8TVX6b7O0oQ==", 2024-01-16T15:01:12.278 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-16T15:01:12.278 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-16T15:01:12.278 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-16T15:01:12.278 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-16T15:01:12.278 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-16T15:01:12.278 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:01:12.278 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:01:12.278 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-16T15:01:12.278 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.b", 2024-01-16T15:01:12.278 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCxmaZlDZ7gGBAAO4XA/MCFRinDyHETOLm6XA==", 2024-01-16T15:01:12.278 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-16T15:01:12.278 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-16T15:01:12.279 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-16T15:01:12.279 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-16T15:01:12.279 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-16T15:01:12.279 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:01:12.279 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:01:12.279 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-16T15:01:12.279 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.c", 2024-01-16T15:01:12.279 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCxmaZleyDIBxAApMtASK3s86++wYNAg2F2wQ==", 2024-01-16T15:01:12.279 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-16T15:01:12.279 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-16T15:01:12.279 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-16T15:01:12.279 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-16T15:01:12.279 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-16T15:01:12.280 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:01:12.280 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:01:12.280 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-16T15:01:12.280 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.d", 2024-01-16T15:01:12.280 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCxmaZllfS2KRAAsrbW5xlokeoer4d86zkoNg==", 2024-01-16T15:01:12.280 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-16T15:01:12.280 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-16T15:01:12.280 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-16T15:01:12.280 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-16T15:01:12.280 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-16T15:01:12.280 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:01:12.280 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:01:12.280 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-16T15:01:12.280 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.0", 2024-01-16T15:01:12.280 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQC2maZl9dWNGRAAvJgRYygFFMUKv+uEdeoQow==", 2024-01-16T15:01:12.281 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-16T15:01:12.281 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-16T15:01:12.281 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-16T15:01:12.281 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-16T15:01:12.281 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:01:12.281 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:01:12.281 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-16T15:01:12.281 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.1", 2024-01-16T15:01:12.281 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQC5maZl0ER+MRAARG3PYOLij8sFyLEAFyKbAg==", 2024-01-16T15:01:12.282 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-16T15:01:12.282 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-16T15:01:12.282 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-16T15:01:12.282 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-16T15:01:12.282 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:01:12.282 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:01:12.282 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-16T15:01:12.282 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.2", 2024-01-16T15:01:12.283 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQC9maZlUCoqDxAA9H2Sg4duizZJ/OtWGIvMoA==", 2024-01-16T15:01:12.283 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-16T15:01:12.283 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-16T15:01:12.283 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-16T15:01:12.283 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-16T15:01:12.283 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:01:12.283 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:01:12.284 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-16T15:01:12.284 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.3", 2024-01-16T15:01:12.284 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDAmaZlnilpGRAAONlgqwADjfzCMCWa7VL1Vw==", 2024-01-16T15:01:12.284 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-16T15:01:12.284 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-16T15:01:12.284 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-16T15:01:12.284 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-16T15:01:12.285 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:01:12.285 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:01:12.285 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-16T15:01:12.285 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.4", 2024-01-16T15:01:12.285 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDHmaZl51rfJRAA9rz9WdAcPwgIOYORvDUy/w==", 2024-01-16T15:01:12.285 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-16T15:01:12.285 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-16T15:01:12.286 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-16T15:01:12.286 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-16T15:01:12.286 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:01:12.286 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:01:12.286 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-16T15:01:12.286 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.5", 2024-01-16T15:01:12.286 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDLmaZlp5daBRAADKUPLUBspcDGWQJgIHhO/g==", 2024-01-16T15:01:12.286 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-16T15:01:12.287 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-16T15:01:12.287 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-16T15:01:12.287 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-16T15:01:12.287 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:01:12.287 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:01:12.287 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-16T15:01:12.287 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.6", 2024-01-16T15:01:12.288 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDOmaZlnBKMDhAAEgUx0WvuThbN7/OVL1symA==", 2024-01-16T15:01:12.288 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-16T15:01:12.288 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-16T15:01:12.288 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-16T15:01:12.288 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-16T15:01:12.288 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:01:12.288 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:01:12.288 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-16T15:01:12.288 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.7", 2024-01-16T15:01:12.288 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDRmaZlD5RUJhAARofymUD17OWQKVXv7LgSQw==", 2024-01-16T15:01:12.288 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-16T15:01:12.288 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-16T15:01:12.288 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-16T15:01:12.289 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-16T15:01:12.289 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:01:12.289 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:01:12.289 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-16T15:01:12.289 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.0", 2024-01-16T15:01:12.289 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCxmaZlyhubNxAANh0FPLUzeCMMZcrjC6cocw==", 2024-01-16T15:01:12.289 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-16T15:01:12.289 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-16T15:01:12.289 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2024-01-16T15:01:12.289 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2024-01-16T15:01:12.289 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2024-01-16T15:01:12.289 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:01:12.289 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:01:12.289 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-16T15:01:12.289 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.1", 2024-01-16T15:01:12.290 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCymaZlO4qRERAA2wjMtfnNF/AHtt4vcfmSlg==", 2024-01-16T15:01:12.290 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-16T15:01:12.290 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-16T15:01:12.290 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2024-01-16T15:01:12.290 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2024-01-16T15:01:12.290 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2024-01-16T15:01:12.290 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:01:12.290 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:01:12.290 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-16T15:01:12.290 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.2", 2024-01-16T15:01:12.290 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCymaZlZnTGBhAAAHMSClKMYlOca2vYv2f5qw==", 2024-01-16T15:01:12.290 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-16T15:01:12.290 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-16T15:01:12.290 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2024-01-16T15:01:12.291 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2024-01-16T15:01:12.291 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2024-01-16T15:01:12.291 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:01:12.291 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:01:12.291 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-16T15:01:12.291 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.3", 2024-01-16T15:01:12.291 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCymaZlJIomHBAASHAHcC+Ks0Nbf4BmJwTcWg==", 2024-01-16T15:01:12.291 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-16T15:01:12.291 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-16T15:01:12.291 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2024-01-16T15:01:12.291 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2024-01-16T15:01:12.291 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2024-01-16T15:01:12.291 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:01:12.291 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:01:12.291 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-16T15:01:12.291 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.admin", 2024-01-16T15:01:12.292 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCvmaZl6K+hJhAAQ1507SpfDqjXieryPHRNSQ==", 2024-01-16T15:01:12.292 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-16T15:01:12.292 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2024-01-16T15:01:12.292 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-16T15:01:12.292 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-16T15:01:12.292 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-16T15:01:12.292 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:01:12.292 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:01:12.292 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-16T15:01:12.292 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.bootstrap-mds", 2024-01-16T15:01:12.292 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDamaZlrbrIHxAART/2C++rl8wgRjJdGBiyCQ==", 2024-01-16T15:01:12.292 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-16T15:01:12.292 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile bootstrap-mds" 2024-01-16T15:01:12.293 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:01:12.293 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:01:12.293 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-16T15:01:12.293 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.bootstrap-mgr", 2024-01-16T15:01:12.293 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDamaZlOcXIHxAATMOg6XqazbXvD4hlwZwrAw==", 2024-01-16T15:01:12.293 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-16T15:01:12.293 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile bootstrap-mgr" 2024-01-16T15:01:12.293 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:01:12.293 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:01:12.293 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-16T15:01:12.293 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.bootstrap-osd", 2024-01-16T15:01:12.293 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDamaZl483IHxAAOQ/xf/YRlpW6t93SkqxB8w==", 2024-01-16T15:01:12.293 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-16T15:01:12.293 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile bootstrap-osd" 2024-01-16T15:01:12.293 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:01:12.294 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:01:12.294 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-16T15:01:12.294 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.bootstrap-rbd", 2024-01-16T15:01:12.294 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDamaZlONjIHxAAs/KOM3qzLGE/41NkozqZNA==", 2024-01-16T15:01:12.294 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-16T15:01:12.294 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile bootstrap-rbd" 2024-01-16T15:01:12.294 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:01:12.294 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:01:12.294 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-16T15:01:12.294 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-01-16T15:01:12.294 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDamaZlAuHIHxAAJeYIyBuLpl+05yDJRqxE2g==", 2024-01-16T15:01:12.294 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-16T15:01:12.294 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-01-16T15:01:12.294 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:01:12.294 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:01:12.294 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-16T15:01:12.295 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.bootstrap-rgw", 2024-01-16T15:01:12.295 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDamaZlt+nIHxAARXqdWxCt3J5kU0T15yPyMw==", 2024-01-16T15:01:12.295 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-16T15:01:12.295 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile bootstrap-rgw" 2024-01-16T15:01:12.295 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:01:12.295 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:01:12.295 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-16T15:01:12.295 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mgr.x", 2024-01-16T15:01:12.295 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCwmaZleViaJxAAh27y3Kew/o1Lf+n25MN9gA==", 2024-01-16T15:01:12.295 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-16T15:01:12.295 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2024-01-16T15:01:12.295 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile mgr", 2024-01-16T15:01:12.295 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-16T15:01:12.295 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:01:12.295 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:01:12.295 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-16T15:01:12.295 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mgr.y", 2024-01-16T15:01:12.296 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCwmaZlwjghHBAA+NmdrcyEDELzm9asEMTSlw==", 2024-01-16T15:01:12.296 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-16T15:01:12.296 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2024-01-16T15:01:12.296 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile mgr", 2024-01-16T15:01:12.296 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-16T15:01:12.296 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:01:12.296 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:01:12.296 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2024-01-16T15:01:12.296 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-16T15:01:12.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.299+0000 7f7882400640 1 -- 172.21.15.143:0/400896961 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f785406b2c0 msgr2=0x7f785406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:12.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.299+0000 7f7882400640 1 --2- 172.21.15.143:0/400896961 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f785406b2c0 0x7f785406d780 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f786c007e90 tx=0x7f786c0073a0 comp rx=0 tx=0).stop 2024-01-16T15:01:12.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.299+0000 7f7882400640 1 -- 172.21.15.143:0/400896961 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f787c0cdfb0 msgr2=0x7f787c15c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:12.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.299+0000 7f7882400640 1 --2- 172.21.15.143:0/400896961 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f787c0cdfb0 0x7f787c15c270 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f786400d800 tx=0x7f786400dcc0 comp rx=0 tx=0).stop 2024-01-16T15:01:12.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.299+0000 7f7882400640 1 -- 172.21.15.143:0/400896961 shutdown_connections 2024-01-16T15:01:12.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.299+0000 7f7882400640 1 --2- 172.21.15.143:0/400896961 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f785406b2c0 0x7f785406d780 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:12.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.299+0000 7f7882400640 1 --2- 172.21.15.143:0/400896961 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f787c15a4f0 0x7f787c15c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:12.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.299+0000 7f7882400640 1 --2- 172.21.15.143:0/400896961 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f787c0cdfb0 0x7f787c15c270 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:12.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.299+0000 7f7882400640 1 --2- 172.21.15.143:0/400896961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f787c0b20c0 0x7f787c09c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:12.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.299+0000 7f7882400640 1 -- 172.21.15.143:0/400896961 >> 172.21.15.143:0/400896961 conn(0x7f787c0b0c70 msgr2=0x7f787c0ade80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:12.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.299+0000 7f7882400640 1 -- 172.21.15.143:0/400896961 shutdown_connections 2024-01-16T15:01:12.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.299+0000 7f7882400640 1 -- 172.21.15.143:0/400896961 wait complete. 2024-01-16T15:01:12.299 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-01-16T15:01:12.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.528+0000 7fc181600640 1 Processor -- start 2024-01-16T15:01:12.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.528+0000 7fc181600640 1 -- start start 2024-01-16T15:01:12.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.530+0000 7fc181600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc17c15b080 0x7fc17c15b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:12.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.530+0000 7fc181600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc17c15e3f0 con 0x7fc17c0b2250 2024-01-16T15:01:12.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.530+0000 7fc181600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc17c15ab80 con 0x7fc17c15df80 2024-01-16T15:01:12.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.530+0000 7fc181600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c15a990 con 0x7fc17c15b080 2024-01-16T15:01:12.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.530+0000 7fc17aa00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc17c0b2250 0x7fc17c0b2630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51700/0 (socket says 172.21.15.143:51700) 2024-01-16T15:01:12.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.530+0000 7fc17aa00640 1 -- 172.21.15.143:0/2003024828 learned_addr learned my addr 172.21.15.143:0/2003024828 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:01:12.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.530+0000 7fc17b400640 1 --2- 172.21.15.143:0/2003024828 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc17c15b080 0x7fc17c15b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:12.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.530+0000 7fc17a000640 1 -- 172.21.15.143:0/2003024828 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4123925393 0 0) 0x7fc17c15e3f0 con 0x7fc17c0b2250 2024-01-16T15:01:12.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.530+0000 7fc17a000640 1 -- 172.21.15.143:0/2003024828 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc15c005730 con 0x7fc17c0b2250 2024-01-16T15:01:12.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.531+0000 7fc17a000640 1 -- 172.21.15.143:0/2003024828 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 991847541 0 0) 0x7fc15c005730 con 0x7fc17c0b2250 2024-01-16T15:01:12.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.531+0000 7fc17a000640 1 -- 172.21.15.143:0/2003024828 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc17c15b080 msgr2=0x7fc17c15b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:12.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.531+0000 7fc17a000640 1 --2- 172.21.15.143:0/2003024828 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc17c15b080 0x7fc17c15b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:12.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.531+0000 7fc17a000640 1 -- 172.21.15.143:0/2003024828 >> v1:172.21.15.204:6789/0 conn(0x7fc17c15df80 legacy=0x7fc17c0ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:12.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.531+0000 7fc17a000640 1 -- 172.21.15.143:0/2003024828 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17c0b2d40 con 0x7fc17c0b2250 2024-01-16T15:01:12.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.531+0000 7fc17a000640 1 -- 172.21.15.143:0/2003024828 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7fc16c001670 con 0x7fc17c0b2250 2024-01-16T15:01:12.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.531+0000 7fc17a000640 1 -- 172.21.15.143:0/2003024828 >> v1:172.21.15.143:6789/0 conn(0x7fc17c0b2250 legacy=0x7fc17c0b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:12.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.531+0000 7fc17a000640 1 --2- 172.21.15.143:0/2003024828 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc17c15b080 0x7fc15c005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:12.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.531+0000 7fc17a000640 1 --2- 172.21.15.143:0/2003024828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc15c006180 0x7fc15c0069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:12.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.531+0000 7fc17a000640 1 --2- 172.21.15.143:0/2003024828 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc15c006f10 0x7fc15c007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:12.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.531+0000 7fc17a000640 1 -- 172.21.15.143:0/2003024828 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c00dca0 con 0x7fc15c006f10 2024-01-16T15:01:12.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.531+0000 7fc17a000640 1 -- 172.21.15.143:0/2003024828 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c00de10 con 0x7fc15c006180 2024-01-16T15:01:12.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.531+0000 7fc17a000640 1 -- 172.21.15.143:0/2003024828 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c00dfd0 con 0x7fc17c15b080 2024-01-16T15:01:12.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.531+0000 7fc17b400640 1 --2- 172.21.15.143:0/2003024828 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc15c006f10 0x7fc15c007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:12.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.532+0000 7fc17aa00640 1 --2- 172.21.15.143:0/2003024828 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc17c15b080 0x7fc15c005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:12.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.532+0000 7fc173e00640 1 --2- 172.21.15.143:0/2003024828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc15c006180 0x7fc15c0069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:12.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.532+0000 7fc17b400640 1 -- 172.21.15.143:0/2003024828 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc17c15b080 msgr2=0x7fc15c005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:12.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.532+0000 7fc17b400640 1 --2- 172.21.15.143:0/2003024828 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc17c15b080 0x7fc15c005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:12.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.532+0000 7fc17b400640 1 -- 172.21.15.143:0/2003024828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc15c006180 msgr2=0x7fc15c0069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:12.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.532+0000 7fc17b400640 1 --2- 172.21.15.143:0/2003024828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc15c006180 0x7fc15c0069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:12.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.532+0000 7fc17b400640 1 -- 172.21.15.143:0/2003024828 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc15c005730 con 0x7fc15c006f10 2024-01-16T15:01:12.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.532+0000 7fc173e00640 1 --2- 172.21.15.143:0/2003024828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc15c006180 0x7fc15c0069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:01:12.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.533+0000 7fc17b400640 1 --2- 172.21.15.143:0/2003024828 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc15c006f10 0x7fc15c007390 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc174009fc0 tx=0x7fc15c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=160647b294816957 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:12.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.533+0000 7fc17a000640 1 -- 172.21.15.143:0/2003024828 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17402cc20 con 0x7fc15c006f10 2024-01-16T15:01:12.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.533+0000 7fc17a000640 1 -- 172.21.15.143:0/2003024828 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17400f040 con 0x7fc15c006f10 2024-01-16T15:01:12.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.533+0000 7fc17a000640 1 -- 172.21.15.143:0/2003024828 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc174034440 con 0x7fc15c006f10 2024-01-16T15:01:12.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.534+0000 7fc181600640 1 -- 172.21.15.143:0/2003024828 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc15c006f10 msgr2=0x7fc15c007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:12.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.534+0000 7fc181600640 1 --2- 172.21.15.143:0/2003024828 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc15c006f10 0x7fc15c007390 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc174009fc0 tx=0x7fc15c007a00 comp rx=0 tx=0).stop 2024-01-16T15:01:12.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.534+0000 7fc17b400640 1 -- 172.21.15.143:0/2003024828 reap_dead start 2024-01-16T15:01:12.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.534+0000 7fc181600640 1 -- 172.21.15.143:0/2003024828 shutdown_connections 2024-01-16T15:01:12.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.534+0000 7fc181600640 1 -- 172.21.15.143:0/2003024828 >> 172.21.15.143:0/2003024828 conn(0x7fc17c06a920 msgr2=0x7fc17c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:12.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.534+0000 7fc181600640 1 -- 172.21.15.143:0/2003024828 shutdown_connections 2024-01-16T15:01:12.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.534+0000 7fc181600640 1 -- 172.21.15.143:0/2003024828 wait complete. 2024-01-16T15:01:12.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.536+0000 7fc181600640 1 Processor -- start 2024-01-16T15:01:12.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.536+0000 7fc181600640 1 -- start start 2024-01-16T15:01:12.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.537+0000 7fc181600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc15c006f10 0x7fc17c0ada60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:12.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.537+0000 7fc181600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc17c0c8360 0x7fc17c150e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:12.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.537+0000 7fc181600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc17c0caa70 0x7fc17c0cf420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:12.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.537+0000 7fc181600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c0c8b90 con 0x7fc17c0caa70 2024-01-16T15:01:12.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.537+0000 7fc181600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c0b1440 con 0x7fc17c0c8360 2024-01-16T15:01:12.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.537+0000 7fc181600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c1517a0 con 0x7fc15c006f10 2024-01-16T15:01:12.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.537+0000 7fc17b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc17c0caa70 0x7fc17c0cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:12.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.537+0000 7fc17b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc17c0caa70 0x7fc17c0cf420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40348/0 (socket says 172.21.15.143:40348) 2024-01-16T15:01:12.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.537+0000 7fc17b400640 1 -- 172.21.15.143:0/555715188 learned_addr learned my addr 172.21.15.143:0/555715188 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:12.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.537+0000 7fc17b400640 1 -- 172.21.15.143:0/555715188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc15c006f10 msgr2=0x7fc17c0ada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:12.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.537+0000 7fc17b400640 1 --2- 172.21.15.143:0/555715188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc15c006f10 0x7fc17c0ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:12.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.537+0000 7fc17b400640 1 -- 172.21.15.143:0/555715188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc17c0c8360 msgr2=0x7fc17c150e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:12.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.537+0000 7fc17b400640 1 --2- 172.21.15.143:0/555715188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc17c0c8360 0x7fc17c150e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:12.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.537+0000 7fc17b400640 1 -- 172.21.15.143:0/555715188 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17c0caef0 con 0x7fc17c0caa70 2024-01-16T15:01:12.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.538+0000 7fc17b400640 1 --2- 172.21.15.143:0/555715188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc17c0caa70 0x7fc17c0cf420 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc174037040 tx=0x7fc174009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:12.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.538+0000 7fc172a00640 1 -- 172.21.15.143:0/555715188 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc174034d50 con 0x7fc17c0caa70 2024-01-16T15:01:12.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.538+0000 7fc172a00640 1 -- 172.21.15.143:0/555715188 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc174003990 con 0x7fc17c0caa70 2024-01-16T15:01:12.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.538+0000 7fc181600640 1 -- 172.21.15.143:0/555715188 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc17c0c88b0 con 0x7fc17c0caa70 2024-01-16T15:01:12.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.538+0000 7fc172a00640 1 -- 172.21.15.143:0/555715188 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc174003d40 con 0x7fc17c0caa70 2024-01-16T15:01:12.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.538+0000 7fc181600640 1 -- 172.21.15.143:0/555715188 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc17c0b1930 con 0x7fc17c0caa70 2024-01-16T15:01:12.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.539+0000 7fc181600640 1 -- 172.21.15.143:0/555715188 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc17c0b1b20 con 0x7fc17c0caa70 2024-01-16T15:01:12.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.540+0000 7fc172a00640 1 -- 172.21.15.143:0/555715188 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fc174004030 con 0x7fc17c0caa70 2024-01-16T15:01:12.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.541+0000 7fc172a00640 1 --2- 172.21.15.143:0/555715188 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fc15406b2c0 0x7fc15406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:12.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.541+0000 7fc172a00640 1 -- 172.21.15.143:0/555715188 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc1740abd80 con 0x7fc17c0caa70 2024-01-16T15:01:12.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.541+0000 7fc17aa00640 1 --2- 172.21.15.143:0/555715188 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fc15406b2c0 0x7fc15406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:12.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.542+0000 7fc17aa00640 1 --2- 172.21.15.143:0/555715188 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fc15406b2c0 0x7fc15406d780 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc16c001050 tx=0x7fc16c000f60 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:12.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.546+0000 7fc172a00640 1 -- 172.21.15.143:0/555715188 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc174077080 con 0x7fc17c0caa70 2024-01-16T15:01:12.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.807+0000 7fc181600640 1 -- 172.21.15.143:0/555715188 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fc17c067470 con 0x7fc17c0caa70 2024-01-16T15:01:12.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.812+0000 7fc172a00640 1 -- 172.21.15.143:0/555715188 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) v1 ==== 90+0+0 (secure 0 0 0) 0x7fc17407a6a0 con 0x7fc17c0caa70 2024-01-16T15:01:12.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.815+0000 7fc181600640 1 -- 172.21.15.143:0/555715188 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fc15406b2c0 msgr2=0x7fc15406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:12.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.815+0000 7fc181600640 1 --2- 172.21.15.143:0/555715188 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fc15406b2c0 0x7fc15406d780 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc16c001050 tx=0x7fc16c000f60 comp rx=0 tx=0).stop 2024-01-16T15:01:12.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.815+0000 7fc181600640 1 -- 172.21.15.143:0/555715188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc17c0caa70 msgr2=0x7fc17c0cf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:12.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.815+0000 7fc181600640 1 --2- 172.21.15.143:0/555715188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc17c0caa70 0x7fc17c0cf420 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc174037040 tx=0x7fc174009750 comp rx=0 tx=0).stop 2024-01-16T15:01:12.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.815+0000 7fc181600640 1 -- 172.21.15.143:0/555715188 shutdown_connections 2024-01-16T15:01:12.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.815+0000 7fc181600640 1 --2- 172.21.15.143:0/555715188 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fc15406b2c0 0x7fc15406d780 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:12.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.815+0000 7fc181600640 1 --2- 172.21.15.143:0/555715188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc17c0caa70 0x7fc17c0cf420 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:12.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.815+0000 7fc181600640 1 --2- 172.21.15.143:0/555715188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc17c0c8360 0x7fc17c150e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:12.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.815+0000 7fc181600640 1 --2- 172.21.15.143:0/555715188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc15c006f10 0x7fc17c0ada60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:12.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.815+0000 7fc181600640 1 -- 172.21.15.143:0/555715188 >> 172.21.15.143:0/555715188 conn(0x7fc17c06a920 msgr2=0x7fc17c157a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:12.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.815+0000 7fc181600640 1 -- 172.21.15.143:0/555715188 shutdown_connections 2024-01-16T15:01:12.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:12.815+0000 7fc181600640 1 -- 172.21.15.143:0/555715188 wait complete. 2024-01-16T15:01:12.813 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-01-16T15:01:13.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.034+0000 7f546f400640 1 Processor -- start 2024-01-16T15:01:13.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.034+0000 7f546f400640 1 -- start start 2024-01-16T15:01:13.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.035+0000 7f546f400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f54680b0b50 0x7f54680b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:13.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.035+0000 7f546f400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f546815e620 0x7f54680ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:13.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.035+0000 7f546f400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f54680ae670 0x7f54680b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:13.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.035+0000 7f546f400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f546815dc10 con 0x7f54680ae670 2024-01-16T15:01:13.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.035+0000 7f546f400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5468072960 con 0x7f54680b0b50 2024-01-16T15:01:13.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.035+0000 7f546f400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54680b1470 con 0x7f546815e620 2024-01-16T15:01:13.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.036+0000 7f546ea00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f54680ae670 0x7f54680b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:13.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.036+0000 7f546ea00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f54680ae670 0x7f54680b1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40358/0 (socket says 172.21.15.143:40358) 2024-01-16T15:01:13.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.036+0000 7f546ea00640 1 -- 172.21.15.143:0/387303001 learned_addr learned my addr 172.21.15.143:0/387303001 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:13.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.036+0000 7f546d600640 1 --2- 172.21.15.143:0/387303001 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f546815e620 0x7f54680ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:13.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.036+0000 7f546e000640 1 --2- 172.21.15.143:0/387303001 >> v2:172.21.15.204:3300/0 conn(0x7f54680b0b50 0x7f54680b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:13.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.036+0000 7f546ea00640 1 -- 172.21.15.143:0/387303001 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f546815e620 msgr2=0x7f54680ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:13.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.036+0000 7f546ea00640 1 --2- 172.21.15.143:0/387303001 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f546815e620 0x7f54680ae130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:13.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.036+0000 7f546ea00640 1 -- 172.21.15.143:0/387303001 >> v2:172.21.15.204:3300/0 conn(0x7f54680b0b50 msgr2=0x7f54680b0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:13.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.036+0000 7f546ea00640 1 --2- 172.21.15.143:0/387303001 >> v2:172.21.15.204:3300/0 conn(0x7f54680b0b50 0x7f54680b0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:13.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.036+0000 7f546ea00640 1 -- 172.21.15.143:0/387303001 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54680cb0b0 con 0x7f54680ae670 2024-01-16T15:01:13.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.036+0000 7f546e000640 1 --2- 172.21.15.143:0/387303001 >> v2:172.21.15.204:3300/0 conn(0x7f54680b0b50 0x7f54680b0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:01:13.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.037+0000 7f546ea00640 1 --2- 172.21.15.143:0/387303001 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54680ae670 0x7f54680b1ec0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f54680b19a0 tx=0x7f546002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f031ba52e0bc1d82 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:13.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.038+0000 7f546cc00640 1 -- 172.21.15.143:0/387303001 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546002ca70 con 0x7f54680ae670 2024-01-16T15:01:13.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.038+0000 7f546cc00640 1 -- 172.21.15.143:0/387303001 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f546002cbd0 con 0x7f54680ae670 2024-01-16T15:01:13.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.038+0000 7f546cc00640 1 -- 172.21.15.143:0/387303001 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54600343f0 con 0x7f54680ae670 2024-01-16T15:01:13.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.038+0000 7f546f400640 1 -- 172.21.15.143:0/387303001 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54680ae670 msgr2=0x7f54680b1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:13.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.038+0000 7f546f400640 1 --2- 172.21.15.143:0/387303001 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54680ae670 0x7f54680b1ec0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f54680b19a0 tx=0x7f546002bed0 comp rx=0 tx=0).stop 2024-01-16T15:01:13.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.039+0000 7f546f400640 1 -- 172.21.15.143:0/387303001 shutdown_connections 2024-01-16T15:01:13.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.039+0000 7f546f400640 1 --2- 172.21.15.143:0/387303001 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54680ae670 0x7f54680b1ec0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:13.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.039+0000 7f546f400640 1 --2- 172.21.15.143:0/387303001 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f546815e620 0x7f54680ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:13.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.039+0000 7f546f400640 1 --2- 172.21.15.143:0/387303001 >> v2:172.21.15.204:3300/0 conn(0x7f54680b0b50 0x7f54680b0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:13.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.039+0000 7f546f400640 1 -- 172.21.15.143:0/387303001 >> 172.21.15.143:0/387303001 conn(0x7f546806a500 msgr2=0x7f546806a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:13.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.039+0000 7f546f400640 1 -- 172.21.15.143:0/387303001 shutdown_connections 2024-01-16T15:01:13.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.039+0000 7f546f400640 1 -- 172.21.15.143:0/387303001 wait complete. 2024-01-16T15:01:13.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.040+0000 7f546f400640 1 Processor -- start 2024-01-16T15:01:13.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.041+0000 7f546f400640 1 -- start start 2024-01-16T15:01:13.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.041+0000 7f546f400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54680ae670 0x7f54681582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:13.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.042+0000 7f546f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54680b0b50 0x7f5468158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:13.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.042+0000 7f546f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f546815e620 0x7f54681f42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:13.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.042+0000 7f546f400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54681589b0 con 0x7f54680b0b50 2024-01-16T15:01:13.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.042+0000 7f546f400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54681593c0 con 0x7f546815e620 2024-01-16T15:01:13.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.042+0000 7f546f400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f546815bc30 con 0x7f54680ae670 2024-01-16T15:01:13.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.042+0000 7f546d600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54680b0b50 0x7f5468158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:13.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.042+0000 7f546d600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54680b0b50 0x7f5468158e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40374/0 (socket says 172.21.15.143:40374) 2024-01-16T15:01:13.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.042+0000 7f546d600640 1 -- 172.21.15.143:0/1195161068 learned_addr learned my addr 172.21.15.143:0/1195161068 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:13.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.042+0000 7f546d600640 1 -- 172.21.15.143:0/1195161068 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54680ae670 msgr2=0x7f54681582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:13.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.042+0000 7f546d600640 1 --2- 172.21.15.143:0/1195161068 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54680ae670 0x7f54681582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:13.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.042+0000 7f546d600640 1 -- 172.21.15.143:0/1195161068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f546815e620 msgr2=0x7f54681f42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:13.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.042+0000 7f546d600640 1 --2- 172.21.15.143:0/1195161068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f546815e620 0x7f54681f42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:13.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.042+0000 7f546d600640 1 -- 172.21.15.143:0/1195161068 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5460009b30 con 0x7f54680b0b50 2024-01-16T15:01:13.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.043+0000 7f546d600640 1 --2- 172.21.15.143:0/1195161068 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54680b0b50 0x7f5468158e80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f545000c7f0 tx=0x7f545000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:13.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.043+0000 7f5466000640 1 -- 172.21.15.143:0/1195161068 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5450004220 con 0x7f54680b0b50 2024-01-16T15:01:13.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.043+0000 7f5466000640 1 -- 172.21.15.143:0/1195161068 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54500043a0 con 0x7f54680b0b50 2024-01-16T15:01:13.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.043+0000 7f5466000640 1 -- 172.21.15.143:0/1195161068 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5450007860 con 0x7f54680b0b50 2024-01-16T15:01:13.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.043+0000 7f546f400640 1 -- 172.21.15.143:0/1195161068 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f546815bf10 con 0x7f54680b0b50 2024-01-16T15:01:13.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.043+0000 7f546f400640 1 -- 172.21.15.143:0/1195161068 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f546815c4d0 con 0x7f54680b0b50 2024-01-16T15:01:13.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.044+0000 7f546f400640 1 -- 172.21.15.143:0/1195161068 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f546815c6c0 con 0x7f54680b0b50 2024-01-16T15:01:13.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.045+0000 7f5466000640 1 -- 172.21.15.143:0/1195161068 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f5450007bd0 con 0x7f54680b0b50 2024-01-16T15:01:13.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.046+0000 7f5466000640 1 --2- 172.21.15.143:0/1195161068 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f544006b2c0 0x7f544006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:13.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.046+0000 7f5466000640 1 -- 172.21.15.143:0/1195161068 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5450089950 con 0x7f54680b0b50 2024-01-16T15:01:13.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.046+0000 7f546e000640 1 --2- 172.21.15.143:0/1195161068 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f544006b2c0 0x7f544006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:13.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.047+0000 7f546e000640 1 --2- 172.21.15.143:0/1195161068 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f544006b2c0 0x7f544006d780 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5458006b10 tx=0x7f5458008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:13.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.051+0000 7f5466000640 1 -- 172.21.15.143:0/1195161068 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5450054a70 con 0x7f54680b0b50 2024-01-16T15:01:13.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.306+0000 7f546f400640 1 -- 172.21.15.143:0/1195161068 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f546804e840 con 0x7f54680b0b50 2024-01-16T15:01:13.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.310+0000 7f5466000640 1 -- 172.21.15.143:0/1195161068 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) v1 ==== 90+0+0 (secure 0 0 0) 0x7f545001e020 con 0x7f54680b0b50 2024-01-16T15:01:13.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.312+0000 7f546f400640 1 -- 172.21.15.143:0/1195161068 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f544006b2c0 msgr2=0x7f544006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:13.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.312+0000 7f546f400640 1 --2- 172.21.15.143:0/1195161068 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f544006b2c0 0x7f544006d780 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5458006b10 tx=0x7f5458008040 comp rx=0 tx=0).stop 2024-01-16T15:01:13.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.312+0000 7f546f400640 1 -- 172.21.15.143:0/1195161068 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54680b0b50 msgr2=0x7f5468158e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:13.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.312+0000 7f546f400640 1 --2- 172.21.15.143:0/1195161068 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54680b0b50 0x7f5468158e80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f545000c7f0 tx=0x7f545000ccb0 comp rx=0 tx=0).stop 2024-01-16T15:01:13.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.312+0000 7f546f400640 1 -- 172.21.15.143:0/1195161068 shutdown_connections 2024-01-16T15:01:13.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.312+0000 7f546f400640 1 --2- 172.21.15.143:0/1195161068 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f544006b2c0 0x7f544006d780 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:13.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.312+0000 7f546f400640 1 --2- 172.21.15.143:0/1195161068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f546815e620 0x7f54681f42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:13.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.312+0000 7f546f400640 1 --2- 172.21.15.143:0/1195161068 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54680b0b50 0x7f5468158e80 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:13.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.312+0000 7f546f400640 1 --2- 172.21.15.143:0/1195161068 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54680ae670 0x7f54681582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:13.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.312+0000 7f546f400640 1 -- 172.21.15.143:0/1195161068 >> 172.21.15.143:0/1195161068 conn(0x7f546806a500 msgr2=0x7f546806a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:13.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.313+0000 7f546f400640 1 -- 172.21.15.143:0/1195161068 shutdown_connections 2024-01-16T15:01:13.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.313+0000 7f546f400640 1 -- 172.21.15.143:0/1195161068 wait complete. 2024-01-16T15:01:13.309 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-01-16T15:01:13.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.526+0000 7fbd9e200640 1 Processor -- start 2024-01-16T15:01:13.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.526+0000 7fbd9e200640 1 -- start start 2024-01-16T15:01:13.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.527+0000 7fbd9e200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd9806aa30 0x7fbd980cc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:13.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.528+0000 7fbd9e200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbd980cf420 0x7fbd980cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:13.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.528+0000 7fbd9e200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd980b2690 con 0x7fbd980b28d0 2024-01-16T15:01:13.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.528+0000 7fbd9e200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd980cc7f0 con 0x7fbd980cf420 2024-01-16T15:01:13.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.528+0000 7fbd9e200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd980cfdf0 con 0x7fbd9806aa30 2024-01-16T15:01:13.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.528+0000 7fbd97e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbd980cf420 0x7fbd980cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:13.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.528+0000 7fbd97400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd9806aa30 0x7fbd980cc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:13.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.528+0000 7fbd97400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd9806aa30 0x7fbd980cc2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:40242/0 (socket says 172.21.15.143:40242) 2024-01-16T15:01:13.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.528+0000 7fbd97400640 1 -- 172.21.15.143:0/3645383510 learned_addr learned my addr 172.21.15.143:0/3645383510 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:13.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.529+0000 7fbd97400640 1 -- 172.21.15.143:0/3645383510 >> v2:172.21.15.204:3300/0 conn(0x7fbd980cf420 msgr2=0x7fbd980cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:13.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.529+0000 7fbd97400640 1 --2- 172.21.15.143:0/3645383510 >> v2:172.21.15.204:3300/0 conn(0x7fbd980cf420 0x7fbd980cf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:13.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.529+0000 7fbd97400640 1 -- 172.21.15.143:0/3645383510 >> v1:172.21.15.204:6789/0 conn(0x7fbd980b28d0 legacy=0x7fbd980cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:13.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.529+0000 7fbd97400640 1 -- 172.21.15.143:0/3645383510 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9806ed00 con 0x7fbd9806aa30 2024-01-16T15:01:13.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.529+0000 7fbd97e00640 1 --2- 172.21.15.143:0/3645383510 >> v2:172.21.15.204:3300/0 conn(0x7fbd980cf420 0x7fbd980cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T15:01:13.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.531+0000 7fbd97400640 1 --2- 172.21.15.143:0/3645383510 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd9806aa30 0x7fbd980cc2b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbd88009e50 tx=0x7fbd8802bff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c221fe5f8359b4 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:13.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.531+0000 7fbd96000640 1 -- 172.21.15.143:0/3645383510 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8802cb90 con 0x7fbd9806aa30 2024-01-16T15:01:13.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.531+0000 7fbd96000640 1 -- 172.21.15.143:0/3645383510 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd8802ccf0 con 0x7fbd9806aa30 2024-01-16T15:01:13.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.531+0000 7fbd96000640 1 -- 172.21.15.143:0/3645383510 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd88034610 con 0x7fbd9806aa30 2024-01-16T15:01:13.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.532+0000 7fbd9e200640 1 -- 172.21.15.143:0/3645383510 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd9806aa30 msgr2=0x7fbd980cc2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:13.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.532+0000 7fbd9e200640 1 --2- 172.21.15.143:0/3645383510 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd9806aa30 0x7fbd980cc2b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbd88009e50 tx=0x7fbd8802bff0 comp rx=0 tx=0).stop 2024-01-16T15:01:13.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.532+0000 7fbd9e200640 1 -- 172.21.15.143:0/3645383510 shutdown_connections 2024-01-16T15:01:13.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.532+0000 7fbd9e200640 1 --2- 172.21.15.143:0/3645383510 >> v2:172.21.15.204:3300/0 conn(0x7fbd980cf420 0x7fbd980cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:13.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.532+0000 7fbd9e200640 1 --2- 172.21.15.143:0/3645383510 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd9806aa30 0x7fbd980cc2b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:13.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.532+0000 7fbd9e200640 1 -- 172.21.15.143:0/3645383510 >> 172.21.15.143:0/3645383510 conn(0x7fbd980999d0 msgr2=0x7fbd980b1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:13.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.532+0000 7fbd9e200640 1 -- 172.21.15.143:0/3645383510 shutdown_connections 2024-01-16T15:01:13.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.532+0000 7fbd9e200640 1 -- 172.21.15.143:0/3645383510 wait complete. 2024-01-16T15:01:13.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.533+0000 7fbd9e200640 1 Processor -- start 2024-01-16T15:01:13.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.534+0000 7fbd9e200640 1 -- start start 2024-01-16T15:01:13.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.534+0000 7fbd9e200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd9806aa30 0x7fbd9814d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:13.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.534+0000 7fbd9e200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd980b28d0 0x7fbd980b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:13.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.534+0000 7fbd9e200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd980cf420 0x7fbd980cb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:13.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.535+0000 7fbd9e200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd980b4300 con 0x7fbd980b28d0 2024-01-16T15:01:13.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.535+0000 7fbd9e200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9814d930 con 0x7fbd9806aa30 2024-01-16T15:01:13.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.535+0000 7fbd9e200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd980b36f0 con 0x7fbd980cf420 2024-01-16T15:01:13.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.535+0000 7fbd96a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd980b28d0 0x7fbd980b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:13.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.535+0000 7fbd96a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd980b28d0 0x7fbd980b2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40386/0 (socket says 172.21.15.143:40386) 2024-01-16T15:01:13.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.535+0000 7fbd96a00640 1 -- 172.21.15.143:0/3383553425 learned_addr learned my addr 172.21.15.143:0/3383553425 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:13.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.535+0000 7fbd97400640 1 --2- 172.21.15.143:0/3383553425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd9806aa30 0x7fbd9814d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:13.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.535+0000 7fbd97e00640 1 --2- 172.21.15.143:0/3383553425 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd980cf420 0x7fbd980cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:13.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.535+0000 7fbd96a00640 1 -- 172.21.15.143:0/3383553425 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd980cf420 msgr2=0x7fbd980cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:13.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.535+0000 7fbd96a00640 1 --2- 172.21.15.143:0/3383553425 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd980cf420 0x7fbd980cb390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:13.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.535+0000 7fbd96a00640 1 -- 172.21.15.143:0/3383553425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd9806aa30 msgr2=0x7fbd9814d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:13.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.535+0000 7fbd96a00640 1 --2- 172.21.15.143:0/3383553425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd9806aa30 0x7fbd9814d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:13.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.535+0000 7fbd96a00640 1 -- 172.21.15.143:0/3383553425 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd90009400 con 0x7fbd980b28d0 2024-01-16T15:01:13.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.536+0000 7fbd96a00640 1 --2- 172.21.15.143:0/3383553425 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd980b28d0 0x7fbd980b2d10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbd8000c400 tx=0x7fbd8000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:13.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.536+0000 7fbd8f400640 1 -- 172.21.15.143:0/3383553425 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd800052d0 con 0x7fbd980b28d0 2024-01-16T15:01:13.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.536+0000 7fbd8f400640 1 -- 172.21.15.143:0/3383553425 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd80005450 con 0x7fbd980b28d0 2024-01-16T15:01:13.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.536+0000 7fbd9e200640 1 -- 172.21.15.143:0/3383553425 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd88009b30 con 0x7fbd980b28d0 2024-01-16T15:01:13.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.536+0000 7fbd8f400640 1 -- 172.21.15.143:0/3383553425 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd80007260 con 0x7fbd980b28d0 2024-01-16T15:01:13.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.536+0000 7fbd9e200640 1 -- 172.21.15.143:0/3383553425 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd980b3cb0 con 0x7fbd980b28d0 2024-01-16T15:01:13.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.537+0000 7fbd9e200640 1 -- 172.21.15.143:0/3383553425 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd980b4470 con 0x7fbd980b28d0 2024-01-16T15:01:13.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.538+0000 7fbd8f400640 1 -- 172.21.15.143:0/3383553425 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fbd800073c0 con 0x7fbd980b28d0 2024-01-16T15:01:13.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.539+0000 7fbd8f400640 1 --2- 172.21.15.143:0/3383553425 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbd7006b2f0 0x7fbd7006d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:13.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.540+0000 7fbd8f400640 1 -- 172.21.15.143:0/3383553425 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbd80088ce0 con 0x7fbd980b28d0 2024-01-16T15:01:13.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.540+0000 7fbd97400640 1 --2- 172.21.15.143:0/3383553425 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbd7006b2f0 0x7fbd7006d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:13.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.540+0000 7fbd97400640 1 --2- 172.21.15.143:0/3383553425 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbd7006b2f0 0x7fbd7006d7b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbd88035000 tx=0x7fbd88030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:13.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.544+0000 7fbd8f400640 1 -- 172.21.15.143:0/3383553425 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbd80053e20 con 0x7fbd980b28d0 2024-01-16T15:01:13.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.799+0000 7fbd9e200640 1 -- 172.21.15.143:0/3383553425 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fbd98067470 con 0x7fbd980b28d0 2024-01-16T15:01:13.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.803+0000 7fbd8f400640 1 -- 172.21.15.143:0/3383553425 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7fbd80057440 con 0x7fbd980b28d0 2024-01-16T15:01:13.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.805+0000 7fbd9e200640 1 -- 172.21.15.143:0/3383553425 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbd7006b2f0 msgr2=0x7fbd7006d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:13.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.805+0000 7fbd9e200640 1 --2- 172.21.15.143:0/3383553425 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbd7006b2f0 0x7fbd7006d7b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbd88035000 tx=0x7fbd88030000 comp rx=0 tx=0).stop 2024-01-16T15:01:13.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.805+0000 7fbd9e200640 1 -- 172.21.15.143:0/3383553425 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd980b28d0 msgr2=0x7fbd980b2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:13.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.805+0000 7fbd9e200640 1 --2- 172.21.15.143:0/3383553425 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd980b28d0 0x7fbd980b2d10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbd8000c400 tx=0x7fbd8000c8c0 comp rx=0 tx=0).stop 2024-01-16T15:01:13.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.805+0000 7fbd9e200640 1 -- 172.21.15.143:0/3383553425 shutdown_connections 2024-01-16T15:01:13.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.805+0000 7fbd9e200640 1 --2- 172.21.15.143:0/3383553425 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbd7006b2f0 0x7fbd7006d7b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:13.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.805+0000 7fbd9e200640 1 --2- 172.21.15.143:0/3383553425 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd980cf420 0x7fbd980cb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:13.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.805+0000 7fbd9e200640 1 --2- 172.21.15.143:0/3383553425 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd980b28d0 0x7fbd980b2d10 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:13.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.805+0000 7fbd9e200640 1 --2- 172.21.15.143:0/3383553425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd9806aa30 0x7fbd9814d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:13.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.805+0000 7fbd9e200640 1 -- 172.21.15.143:0/3383553425 >> 172.21.15.143:0/3383553425 conn(0x7fbd980999d0 msgr2=0x7fbd980af0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:13.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.805+0000 7fbd9e200640 1 -- 172.21.15.143:0/3383553425 shutdown_connections 2024-01-16T15:01:13.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:13.805+0000 7fbd9e200640 1 -- 172.21.15.143:0/3383553425 wait complete. 2024-01-16T15:01:13.803 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-01-16T15:01:13.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.000+0000 7f254aa00640 1 Processor -- start 2024-01-16T15:01:13.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.001+0000 7f254aa00640 1 -- start start 2024-01-16T15:01:13.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.002+0000 7f254aa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25440cfb60 0x7f2544161df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:13.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.002+0000 7f254aa00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25440ce0b0 con 0x7f254415dc80 2024-01-16T15:01:13.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.002+0000 7f254aa00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2544161960 con 0x7f2544157c70 2024-01-16T15:01:13.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.002+0000 7f254aa00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f254415e8d0 con 0x7f25440cfb60 2024-01-16T15:01:13.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.002+0000 7f2548a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f254415dc80 0x7f25440b31b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51708/0 (socket says 172.21.15.143:51708) 2024-01-16T15:01:13.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.002+0000 7f2548a00640 1 -- 172.21.15.143:0/1090494444 learned_addr learned my addr 172.21.15.143:0/1090494444 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:01:13.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.003+0000 7f253b200640 1 --2- 172.21.15.143:0/1090494444 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25440cfb60 0x7f2544161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:13.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.003+0000 7f2543400640 1 -- 172.21.15.143:0/1090494444 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 918202452 0 0) 0x7f25440ce0b0 con 0x7f254415dc80 2024-01-16T15:01:13.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.003+0000 7f2543400640 1 -- 172.21.15.143:0/1090494444 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2524003520 con 0x7f254415dc80 2024-01-16T15:01:13.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.003+0000 7f2543400640 1 -- 172.21.15.143:0/1090494444 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1075637931 0 0) 0x7f2524003520 con 0x7f254415dc80 2024-01-16T15:01:13.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.003+0000 7f2543400640 1 -- 172.21.15.143:0/1090494444 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25440cfb60 msgr2=0x7f2544161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:13.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.003+0000 7f2543400640 1 --2- 172.21.15.143:0/1090494444 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25440cfb60 0x7f2544161df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:13.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.003+0000 7f2543400640 1 -- 172.21.15.143:0/1090494444 >> v1:172.21.15.204:6789/0 conn(0x7f2544157c70 legacy=0x7f25440cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:13.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.003+0000 7f2543400640 1 -- 172.21.15.143:0/1090494444 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f254415ebb0 con 0x7f254415dc80 2024-01-16T15:01:13.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.003+0000 7f2543400640 1 -- 172.21.15.143:0/1090494444 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7f253c0019f0 con 0x7f254415dc80 2024-01-16T15:01:13.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.003+0000 7f2543400640 1 -- 172.21.15.143:0/1090494444 >> v1:172.21.15.143:6789/0 conn(0x7f254415dc80 legacy=0x7f25440b31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:13.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.003+0000 7f2543400640 1 --2- 172.21.15.143:0/1090494444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25240039a0 0x7f2524003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:13.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.004+0000 7f2543400640 1 --2- 172.21.15.143:0/1090494444 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25440cfb60 0x7f25240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:13.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.004+0000 7f2543400640 1 --2- 172.21.15.143:0/1090494444 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2524004d00 0x7f2524005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:13.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.004+0000 7f2543400640 1 -- 172.21.15.143:0/1090494444 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254415e8d0 con 0x7f2524004d00 2024-01-16T15:01:13.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.004+0000 7f2543400640 1 -- 172.21.15.143:0/1090494444 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252400bab0 con 0x7f25240039a0 2024-01-16T15:01:13.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.004+0000 7f2543400640 1 -- 172.21.15.143:0/1090494444 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f252400bc70 con 0x7f25440cfb60 2024-01-16T15:01:13.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.004+0000 7f2548a00640 1 --2- 172.21.15.143:0/1090494444 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2524004d00 0x7f2524005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:13.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.004+0000 7f2543e00640 1 --2- 172.21.15.143:0/1090494444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25240039a0 0x7f2524003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:13.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.004+0000 7f253b200640 1 --2- 172.21.15.143:0/1090494444 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25440cfb60 0x7f25240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:13.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.004+0000 7f2548a00640 1 -- 172.21.15.143:0/1090494444 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25440cfb60 msgr2=0x7f25240047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:13.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.004+0000 7f2548a00640 1 --2- 172.21.15.143:0/1090494444 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25440cfb60 0x7f25240047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:13.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.004+0000 7f2548a00640 1 -- 172.21.15.143:0/1090494444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25240039a0 msgr2=0x7f2524003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:13.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.004+0000 7f2548a00640 1 --2- 172.21.15.143:0/1090494444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25240039a0 0x7f2524003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:13.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.004+0000 7f2548a00640 1 -- 172.21.15.143:0/1090494444 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2524003520 con 0x7f2524004d00 2024-01-16T15:01:13.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.004+0000 7f2543e00640 1 --2- 172.21.15.143:0/1090494444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25240039a0 0x7f2524003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:01:13.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.004+0000 7f253b200640 1 --2- 172.21.15.143:0/1090494444 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25440cfb60 0x7f25240047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:01:13.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.005+0000 7f2548a00640 1 --2- 172.21.15.143:0/1090494444 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2524004d00 0x7f2524005180 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f253c00a830 tx=0x7f25240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f766e92b2d3e05 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:13.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.006+0000 7f2543400640 1 -- 172.21.15.143:0/1090494444 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253c0072d0 con 0x7f2524004d00 2024-01-16T15:01:13.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.006+0000 7f2543400640 1 -- 172.21.15.143:0/1090494444 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f253c02f070 con 0x7f2524004d00 2024-01-16T15:01:13.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.006+0000 7f2543400640 1 -- 172.21.15.143:0/1090494444 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253c0076b0 con 0x7f2524004d00 2024-01-16T15:01:13.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.006+0000 7f254aa00640 1 -- 172.21.15.143:0/1090494444 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2524004d00 msgr2=0x7f2524005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:13.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.006+0000 7f254aa00640 1 --2- 172.21.15.143:0/1090494444 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2524004d00 0x7f2524005180 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f253c00a830 tx=0x7f25240057f0 comp rx=0 tx=0).stop 2024-01-16T15:01:13.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.006+0000 7f2548a00640 1 -- 172.21.15.143:0/1090494444 reap_dead start 2024-01-16T15:01:13.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.006+0000 7f254aa00640 1 -- 172.21.15.143:0/1090494444 shutdown_connections 2024-01-16T15:01:13.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.006+0000 7f254aa00640 1 -- 172.21.15.143:0/1090494444 >> 172.21.15.143:0/1090494444 conn(0x7f254406a920 msgr2=0x7f254406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:13.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.006+0000 7f254aa00640 1 -- 172.21.15.143:0/1090494444 shutdown_connections 2024-01-16T15:01:13.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.006+0000 7f254aa00640 1 -- 172.21.15.143:0/1090494444 wait complete. 2024-01-16T15:01:13.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.008+0000 7f254aa00640 1 Processor -- start 2024-01-16T15:01:13.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.008+0000 7f254aa00640 1 -- start start 2024-01-16T15:01:13.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.009+0000 7f254aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2524004d00 0x7f254414b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:13.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.009+0000 7f254aa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f254409c0b0 0x7f25441f36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:13.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.009+0000 7f254aa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25441f3c10 0x7f254409a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:13.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.009+0000 7f254aa00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254414b9d0 con 0x7f2524004d00 2024-01-16T15:01:13.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.009+0000 7f254aa00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25440996f0 con 0x7f254409c0b0 2024-01-16T15:01:13.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.009+0000 7f254aa00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25440999d0 con 0x7f25441f3c10 2024-01-16T15:01:13.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.009+0000 7f2543e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2524004d00 0x7f254414b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:13.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.009+0000 7f2543e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2524004d00 0x7f254414b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40404/0 (socket says 172.21.15.143:40404) 2024-01-16T15:01:13.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.009+0000 7f2543e00640 1 -- 172.21.15.143:0/1036718914 learned_addr learned my addr 172.21.15.143:0/1036718914 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:13.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.009+0000 7f2543e00640 1 -- 172.21.15.143:0/1036718914 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25441f3c10 msgr2=0x7f254409a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:13.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.009+0000 7f253b200640 1 --2- 172.21.15.143:0/1036718914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f254409c0b0 0x7f25441f36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:13.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.009+0000 7f2543e00640 1 --2- 172.21.15.143:0/1036718914 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25441f3c10 0x7f254409a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:13.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.009+0000 7f2543e00640 1 -- 172.21.15.143:0/1036718914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f254409c0b0 msgr2=0x7f25441f36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:13.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.010+0000 7f2543e00640 1 --2- 172.21.15.143:0/1036718914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f254409c0b0 0x7f25441f36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:13.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.010+0000 7f2543e00640 1 -- 172.21.15.143:0/1036718914 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2544099cb0 con 0x7f2524004d00 2024-01-16T15:01:13.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.010+0000 7f2543e00640 1 --2- 172.21.15.143:0/1036718914 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2524004d00 0x7f254414b490 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f253400d720 tx=0x7f253400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:13.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.010+0000 7f2540c00640 1 -- 172.21.15.143:0/1036718914 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2534017070 con 0x7f2524004d00 2024-01-16T15:01:13.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.010+0000 7f2540c00640 1 -- 172.21.15.143:0/1036718914 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2534005070 con 0x7f2524004d00 2024-01-16T15:01:13.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.010+0000 7f2540c00640 1 -- 172.21.15.143:0/1036718914 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2534005420 con 0x7f2524004d00 2024-01-16T15:01:13.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.010+0000 7f254aa00640 1 -- 172.21.15.143:0/1036718914 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2544099f90 con 0x7f2524004d00 2024-01-16T15:01:13.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.010+0000 7f254aa00640 1 -- 172.21.15.143:0/1036718914 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f254409c530 con 0x7f2524004d00 2024-01-16T15:01:13.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.011+0000 7f254aa00640 1 -- 172.21.15.143:0/1036718914 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2544067470 con 0x7f2524004d00 2024-01-16T15:01:13.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.012+0000 7f2540c00640 1 -- 172.21.15.143:0/1036718914 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f2534007030 con 0x7f2524004d00 2024-01-16T15:01:13.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.013+0000 7f2540c00640 1 --2- 172.21.15.143:0/1036718914 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f251c06b2f0 0x7f251c06d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:13.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.014+0000 7f2540c00640 1 -- 172.21.15.143:0/1036718914 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2534089120 con 0x7f2524004d00 2024-01-16T15:01:13.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.014+0000 7f253b200640 1 --2- 172.21.15.143:0/1036718914 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f251c06b2f0 0x7f251c06d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:13.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.014+0000 7f253b200640 1 --2- 172.21.15.143:0/1036718914 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f251c06b2f0 0x7f251c06d7b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f25440b29e0 tx=0x7f252c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:13.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.018+0000 7f2540c00640 1 -- 172.21.15.143:0/1036718914 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f25340542d0 con 0x7f2524004d00 2024-01-16T15:01:14.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.255+0000 7f254aa00640 1 -- 172.21.15.143:0/1036718914 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f254406ef20 con 0x7f2524004d00 2024-01-16T15:01:14.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.260+0000 7f2540c00640 1 -- 172.21.15.143:0/1036718914 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f25340578f0 con 0x7f2524004d00 2024-01-16T15:01:14.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.262+0000 7f254aa00640 1 -- 172.21.15.143:0/1036718914 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f251c06b2f0 msgr2=0x7f251c06d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:14.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.262+0000 7f254aa00640 1 --2- 172.21.15.143:0/1036718914 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f251c06b2f0 0x7f251c06d7b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f25440b29e0 tx=0x7f252c008040 comp rx=0 tx=0).stop 2024-01-16T15:01:14.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.262+0000 7f254aa00640 1 -- 172.21.15.143:0/1036718914 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2524004d00 msgr2=0x7f254414b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:14.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.262+0000 7f254aa00640 1 --2- 172.21.15.143:0/1036718914 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2524004d00 0x7f254414b490 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f253400d720 tx=0x7f253400dbe0 comp rx=0 tx=0).stop 2024-01-16T15:01:14.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.262+0000 7f254aa00640 1 -- 172.21.15.143:0/1036718914 shutdown_connections 2024-01-16T15:01:14.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.262+0000 7f254aa00640 1 --2- 172.21.15.143:0/1036718914 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f251c06b2f0 0x7f251c06d7b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:14.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.262+0000 7f254aa00640 1 --2- 172.21.15.143:0/1036718914 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25441f3c10 0x7f254409a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:14.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.262+0000 7f254aa00640 1 --2- 172.21.15.143:0/1036718914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f254409c0b0 0x7f25441f36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:14.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.262+0000 7f254aa00640 1 --2- 172.21.15.143:0/1036718914 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2524004d00 0x7f254414b490 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:14.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.262+0000 7f254aa00640 1 -- 172.21.15.143:0/1036718914 >> 172.21.15.143:0/1036718914 conn(0x7f254406a920 msgr2=0x7f25440adfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:14.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.262+0000 7f254aa00640 1 -- 172.21.15.143:0/1036718914 shutdown_connections 2024-01-16T15:01:14.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.263+0000 7f254aa00640 1 -- 172.21.15.143:0/1036718914 wait complete. 2024-01-16T15:01:14.259 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-01-16T15:01:14.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.483+0000 7f9897800640 1 Processor -- start 2024-01-16T15:01:14.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.483+0000 7f9897800640 1 -- start start 2024-01-16T15:01:14.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.485+0000 7f9897800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9890072160 0x7f989015e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:14.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.485+0000 7f9897800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98900c49f0 0x7f989006d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:14.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.485+0000 7f9897800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f989006dc40 0x7f98900cd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:14.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.485+0000 7f9897800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f989006e0f0 con 0x7f989006dc40 2024-01-16T15:01:14.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.485+0000 7f9897800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f989015ec90 con 0x7f9890072160 2024-01-16T15:01:14.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.485+0000 7f9897800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98900cd530 con 0x7f98900c49f0 2024-01-16T15:01:14.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.485+0000 7f9895800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f989006dc40 0x7f98900cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:14.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.485+0000 7f9895800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f989006dc40 0x7f98900cd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40414/0 (socket says 172.21.15.143:40414) 2024-01-16T15:01:14.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.485+0000 7f9895800640 1 -- 172.21.15.143:0/3486998013 learned_addr learned my addr 172.21.15.143:0/3486998013 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:14.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.486+0000 7f9895800640 1 -- 172.21.15.143:0/3486998013 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98900c49f0 msgr2=0x7f989006d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:14.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.485+0000 7f988fe00640 1 --2- 172.21.15.143:0/3486998013 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98900c49f0 0x7f989006d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:14.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.486+0000 7f9894e00640 1 --2- 172.21.15.143:0/3486998013 >> v2:172.21.15.204:3300/0 conn(0x7f9890072160 0x7f989015e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:14.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.486+0000 7f9895800640 1 --2- 172.21.15.143:0/3486998013 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98900c49f0 0x7f989006d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:14.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.486+0000 7f9895800640 1 -- 172.21.15.143:0/3486998013 >> v2:172.21.15.204:3300/0 conn(0x7f9890072160 msgr2=0x7f989015e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:14.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.486+0000 7f9895800640 1 --2- 172.21.15.143:0/3486998013 >> v2:172.21.15.204:3300/0 conn(0x7f9890072160 0x7f989015e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:14.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.486+0000 7f9895800640 1 -- 172.21.15.143:0/3486998013 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98900c4ed0 con 0x7f989006dc40 2024-01-16T15:01:14.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.486+0000 7f9894e00640 1 --2- 172.21.15.143:0/3486998013 >> v2:172.21.15.204:3300/0 conn(0x7f9890072160 0x7f989015e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:01:14.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.487+0000 7f9895800640 1 --2- 172.21.15.143:0/3486998013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f989006dc40 0x7f98900cd750 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f98900ccae0 tx=0x7f988002db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f17d5510f0e7abc server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:14.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.487+0000 7f988f400640 1 -- 172.21.15.143:0/3486998013 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988002f8d0 con 0x7f989006dc40 2024-01-16T15:01:14.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.487+0000 7f988f400640 1 -- 172.21.15.143:0/3486998013 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988002fa30 con 0x7f989006dc40 2024-01-16T15:01:14.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.487+0000 7f988f400640 1 -- 172.21.15.143:0/3486998013 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988002fdc0 con 0x7f989006dc40 2024-01-16T15:01:14.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.488+0000 7f9897800640 1 -- 172.21.15.143:0/3486998013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f989006dc40 msgr2=0x7f98900cd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:14.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.488+0000 7f9897800640 1 --2- 172.21.15.143:0/3486998013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f989006dc40 0x7f98900cd750 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f98900ccae0 tx=0x7f988002db10 comp rx=0 tx=0).stop 2024-01-16T15:01:14.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.488+0000 7f9897800640 1 -- 172.21.15.143:0/3486998013 shutdown_connections 2024-01-16T15:01:14.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.488+0000 7f9897800640 1 --2- 172.21.15.143:0/3486998013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f989006dc40 0x7f98900cd750 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:14.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.488+0000 7f9897800640 1 --2- 172.21.15.143:0/3486998013 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98900c49f0 0x7f989006d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:14.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.488+0000 7f9897800640 1 --2- 172.21.15.143:0/3486998013 >> v2:172.21.15.204:3300/0 conn(0x7f9890072160 0x7f989015e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:14.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.488+0000 7f9897800640 1 -- 172.21.15.143:0/3486998013 >> 172.21.15.143:0/3486998013 conn(0x7f98900b0c50 msgr2=0x7f98900b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:14.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.488+0000 7f9897800640 1 -- 172.21.15.143:0/3486998013 shutdown_connections 2024-01-16T15:01:14.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.488+0000 7f9897800640 1 -- 172.21.15.143:0/3486998013 wait complete. 2024-01-16T15:01:14.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.490+0000 7f9897800640 1 Processor -- start 2024-01-16T15:01:14.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.490+0000 7f9897800640 1 -- start start 2024-01-16T15:01:14.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.491+0000 7f9897800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f989006dc40 0x7f98901f37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:14.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.491+0000 7f9897800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9890072160 0x7f98900ac1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:14.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.491+0000 7f9897800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98900c49f0 0x7f9890151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:14.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.492+0000 7f9897800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98901f3d00 con 0x7f98900c49f0 2024-01-16T15:01:14.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.492+0000 7f9897800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989015a470 con 0x7f9890072160 2024-01-16T15:01:14.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.492+0000 7f9897800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98900c7120 con 0x7f989006dc40 2024-01-16T15:01:14.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.492+0000 7f9895800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98900c49f0 0x7f9890151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:14.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.492+0000 7f9895800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98900c49f0 0x7f9890151e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40424/0 (socket says 172.21.15.143:40424) 2024-01-16T15:01:14.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.492+0000 7f9895800640 1 -- 172.21.15.143:0/1113175372 learned_addr learned my addr 172.21.15.143:0/1113175372 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:14.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.492+0000 7f9894e00640 1 --2- 172.21.15.143:0/1113175372 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f989006dc40 0x7f98901f37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:14.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.492+0000 7f9895800640 1 -- 172.21.15.143:0/1113175372 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f989006dc40 msgr2=0x7f98901f37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:14.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.492+0000 7f988fe00640 1 --2- 172.21.15.143:0/1113175372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9890072160 0x7f98900ac1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:14.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.492+0000 7f9895800640 1 --2- 172.21.15.143:0/1113175372 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f989006dc40 0x7f98901f37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:14.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.492+0000 7f9895800640 1 -- 172.21.15.143:0/1113175372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9890072160 msgr2=0x7f98900ac1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:14.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.492+0000 7f9895800640 1 --2- 172.21.15.143:0/1113175372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9890072160 0x7f98900ac1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:14.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.492+0000 7f9895800640 1 -- 172.21.15.143:0/1113175372 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98800094d0 con 0x7f98900c49f0 2024-01-16T15:01:14.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.492+0000 7f9895800640 1 --2- 172.21.15.143:0/1113175372 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98900c49f0 0x7f9890151e30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f98800090b0 tx=0x7f98800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:14.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.493+0000 7f988cc00640 1 -- 172.21.15.143:0/1113175372 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9880037ca0 con 0x7f98900c49f0 2024-01-16T15:01:14.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.493+0000 7f988cc00640 1 -- 172.21.15.143:0/1113175372 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9880037e00 con 0x7f98900c49f0 2024-01-16T15:01:14.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.493+0000 7f9897800640 1 -- 172.21.15.143:0/1113175372 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98900c7370 con 0x7f98900c49f0 2024-01-16T15:01:14.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.493+0000 7f988cc00640 1 -- 172.21.15.143:0/1113175372 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98800426b0 con 0x7f98900c49f0 2024-01-16T15:01:14.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.493+0000 7f9897800640 1 -- 172.21.15.143:0/1113175372 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98900c7c10 con 0x7f98900c49f0 2024-01-16T15:01:14.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.494+0000 7f9897800640 1 -- 172.21.15.143:0/1113175372 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f989004e8b0 con 0x7f98900c49f0 2024-01-16T15:01:14.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.495+0000 7f988cc00640 1 -- 172.21.15.143:0/1113175372 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f9880042810 con 0x7f98900c49f0 2024-01-16T15:01:14.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.496+0000 7f988cc00640 1 --2- 172.21.15.143:0/1113175372 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f986806b2c0 0x7f986806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:14.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.496+0000 7f988cc00640 1 -- 172.21.15.143:0/1113175372 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9880033070 con 0x7f98900c49f0 2024-01-16T15:01:14.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.496+0000 7f9894e00640 1 --2- 172.21.15.143:0/1113175372 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f986806b2c0 0x7f986806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:14.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.497+0000 7f9894e00640 1 --2- 172.21.15.143:0/1113175372 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f986806b2c0 0x7f986806d780 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f98900acac0 tx=0x7f9888008470 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:14.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.500+0000 7f988cc00640 1 -- 172.21.15.143:0/1113175372 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9880077b20 con 0x7f98900c49f0 2024-01-16T15:01:14.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.758+0000 7f9897800640 1 -- 172.21.15.143:0/1113175372 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f989015dab0 con 0x7f98900c49f0 2024-01-16T15:01:14.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.763+0000 7f988cc00640 1 -- 172.21.15.143:0/1113175372 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7f988007b140 con 0x7f98900c49f0 2024-01-16T15:01:14.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.765+0000 7f9897800640 1 -- 172.21.15.143:0/1113175372 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f986806b2c0 msgr2=0x7f986806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:14.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.765+0000 7f9897800640 1 --2- 172.21.15.143:0/1113175372 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f986806b2c0 0x7f986806d780 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f98900acac0 tx=0x7f9888008470 comp rx=0 tx=0).stop 2024-01-16T15:01:14.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.765+0000 7f9897800640 1 -- 172.21.15.143:0/1113175372 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98900c49f0 msgr2=0x7f9890151e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:14.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.765+0000 7f9897800640 1 --2- 172.21.15.143:0/1113175372 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98900c49f0 0x7f9890151e30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f98800090b0 tx=0x7f98800046c0 comp rx=0 tx=0).stop 2024-01-16T15:01:14.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.766+0000 7f9897800640 1 -- 172.21.15.143:0/1113175372 shutdown_connections 2024-01-16T15:01:14.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.766+0000 7f9897800640 1 --2- 172.21.15.143:0/1113175372 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f986806b2c0 0x7f986806d780 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:14.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.766+0000 7f9897800640 1 --2- 172.21.15.143:0/1113175372 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98900c49f0 0x7f9890151e30 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:14.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.766+0000 7f9897800640 1 --2- 172.21.15.143:0/1113175372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9890072160 0x7f98900ac1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:14.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.766+0000 7f9897800640 1 --2- 172.21.15.143:0/1113175372 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f989006dc40 0x7f98901f37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:14.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.766+0000 7f9897800640 1 -- 172.21.15.143:0/1113175372 >> 172.21.15.143:0/1113175372 conn(0x7f98900b0c50 msgr2=0x7f989015e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:14.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.766+0000 7f9897800640 1 -- 172.21.15.143:0/1113175372 shutdown_connections 2024-01-16T15:01:14.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.766+0000 7f9897800640 1 -- 172.21.15.143:0/1113175372 wait complete. 2024-01-16T15:01:14.763 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-01-16T15:01:14.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.994+0000 7fd11ac00640 1 Processor -- start 2024-01-16T15:01:14.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.994+0000 7fd11ac00640 1 -- start start 2024-01-16T15:01:14.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.996+0000 7fd11ac00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd11415e620 0x7fd1140ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:14.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.996+0000 7fd11ac00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd1140ae670 0x7fd1140b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:14.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.996+0000 7fd11ac00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd114072960 con 0x7fd1140b0b50 2024-01-16T15:01:14.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.996+0000 7fd11ac00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd11415dc10 con 0x7fd1140ae670 2024-01-16T15:01:14.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.996+0000 7fd11ac00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1140cb0b0 con 0x7fd11415e620 2024-01-16T15:01:14.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.996+0000 7fd11a200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd1140ae670 0x7fd1140b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:14.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.996+0000 7fd11a200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd1140ae670 0x7fd1140b1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58966/0 (socket says 172.21.15.143:58966) 2024-01-16T15:01:14.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.996+0000 7fd11a200640 1 -- 172.21.15.143:0/1169264073 learned_addr learned my addr 172.21.15.143:0/1169264073 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:14.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.996+0000 7fd118e00640 1 --2- 172.21.15.143:0/1169264073 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd11415e620 0x7fd1140ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:14.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.996+0000 7fd11a200640 1 -- 172.21.15.143:0/1169264073 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd11415e620 msgr2=0x7fd1140ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:14.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.996+0000 7fd11a200640 1 --2- 172.21.15.143:0/1169264073 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd11415e620 0x7fd1140ae130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:14.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.996+0000 7fd11a200640 1 -- 172.21.15.143:0/1169264073 >> v1:172.21.15.204:6789/0 conn(0x7fd1140b0b50 legacy=0x7fd1140b0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:14.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.997+0000 7fd11a200640 1 -- 172.21.15.143:0/1169264073 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1140cb390 con 0x7fd1140ae670 2024-01-16T15:01:14.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.998+0000 7fd11a200640 1 --2- 172.21.15.143:0/1169264073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1140ae670 0x7fd1140b1ec0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd10c009e50 tx=0x7fd10c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=239787d5544fea60 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:14.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.998+0000 7fd113e00640 1 -- 172.21.15.143:0/1169264073 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd10c02ca80 con 0x7fd1140ae670 2024-01-16T15:01:14.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.998+0000 7fd113e00640 1 -- 172.21.15.143:0/1169264073 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd10c02cbe0 con 0x7fd1140ae670 2024-01-16T15:01:14.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.998+0000 7fd113e00640 1 -- 172.21.15.143:0/1169264073 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd10c0343f0 con 0x7fd1140ae670 2024-01-16T15:01:14.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.999+0000 7fd11ac00640 1 -- 172.21.15.143:0/1169264073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1140ae670 msgr2=0x7fd1140b1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:14.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.999+0000 7fd11ac00640 1 --2- 172.21.15.143:0/1169264073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1140ae670 0x7fd1140b1ec0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd10c009e50 tx=0x7fd10c02bee0 comp rx=0 tx=0).stop 2024-01-16T15:01:14.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.999+0000 7fd11ac00640 1 -- 172.21.15.143:0/1169264073 shutdown_connections 2024-01-16T15:01:14.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.999+0000 7fd11ac00640 1 --2- 172.21.15.143:0/1169264073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1140ae670 0x7fd1140b1ec0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:14.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.999+0000 7fd11ac00640 1 --2- 172.21.15.143:0/1169264073 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd11415e620 0x7fd1140ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:14.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.999+0000 7fd11ac00640 1 -- 172.21.15.143:0/1169264073 >> 172.21.15.143:0/1169264073 conn(0x7fd11406a500 msgr2=0x7fd11406a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:14.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.999+0000 7fd11ac00640 1 -- 172.21.15.143:0/1169264073 shutdown_connections 2024-01-16T15:01:14.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:14.999+0000 7fd11ac00640 1 -- 172.21.15.143:0/1169264073 wait complete. 2024-01-16T15:01:14.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.001+0000 7fd11ac00640 1 Processor -- start 2024-01-16T15:01:14.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.001+0000 7fd11ac00640 1 -- start start 2024-01-16T15:01:14.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.001+0000 7fd11ac00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1140ae670 0x7fd114155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:14.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.002+0000 7fd11ac00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1140b0b50 0x7fd11414f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:14.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.002+0000 7fd11ac00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd11415e620 0x7fd11414e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:14.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.002+0000 7fd11ac00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11414fe50 con 0x7fd11415e620 2024-01-16T15:01:14.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.002+0000 7fd11ac00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd114153190 con 0x7fd1140ae670 2024-01-16T15:01:14.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.002+0000 7fd11ac00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd11415c090 con 0x7fd1140b0b50 2024-01-16T15:01:14.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.002+0000 7fd11a200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd11415e620 0x7fd11414e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:14.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.002+0000 7fd11a200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd11415e620 0x7fd11414e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58972/0 (socket says 172.21.15.143:58972) 2024-01-16T15:01:14.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.002+0000 7fd11a200640 1 -- 172.21.15.143:0/3543001393 learned_addr learned my addr 172.21.15.143:0/3543001393 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:14.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.002+0000 7fd11a200640 1 -- 172.21.15.143:0/3543001393 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1140b0b50 msgr2=0x7fd11414f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:14.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.002+0000 7fd11a200640 1 --2- 172.21.15.143:0/3543001393 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1140b0b50 0x7fd11414f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:14.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.002+0000 7fd119800640 1 --2- 172.21.15.143:0/3543001393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1140ae670 0x7fd114155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:14.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.002+0000 7fd11a200640 1 -- 172.21.15.143:0/3543001393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1140ae670 msgr2=0x7fd114155400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:14.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.002+0000 7fd11a200640 1 --2- 172.21.15.143:0/3543001393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1140ae670 0x7fd114155400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:14.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.002+0000 7fd11a200640 1 -- 172.21.15.143:0/3543001393 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd10c009b30 con 0x7fd11415e620 2024-01-16T15:01:14.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.003+0000 7fd11a200640 1 --2- 172.21.15.143:0/3543001393 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd11415e620 0x7fd11414e880 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd10c036040 tx=0x7fd10c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:14.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.003+0000 7fd111600640 1 -- 172.21.15.143:0/3543001393 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd10c009510 con 0x7fd11415e620 2024-01-16T15:01:14.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.003+0000 7fd111600640 1 -- 172.21.15.143:0/3543001393 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd10c03e450 con 0x7fd11415e620 2024-01-16T15:01:14.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.003+0000 7fd111600640 1 -- 172.21.15.143:0/3543001393 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd10c03e800 con 0x7fd11415e620 2024-01-16T15:01:14.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.003+0000 7fd11ac00640 1 -- 172.21.15.143:0/3543001393 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1141533e0 con 0x7fd11415e620 2024-01-16T15:01:14.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.003+0000 7fd11ac00640 1 -- 172.21.15.143:0/3543001393 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd11415bd20 con 0x7fd11415e620 2024-01-16T15:01:14.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.004+0000 7fd11ac00640 1 -- 172.21.15.143:0/3543001393 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd11415b8a0 con 0x7fd11415e620 2024-01-16T15:01:14.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.005+0000 7fd111600640 1 -- 172.21.15.143:0/3543001393 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fd10c02cd50 con 0x7fd11415e620 2024-01-16T15:01:14.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.006+0000 7fd111600640 1 --2- 172.21.15.143:0/3543001393 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd0ec06b2c0 0x7fd0ec06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:14.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.006+0000 7fd111600640 1 -- 172.21.15.143:0/3543001393 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd10c0ac470 con 0x7fd11415e620 2024-01-16T15:01:14.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.006+0000 7fd119800640 1 --2- 172.21.15.143:0/3543001393 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd0ec06b2c0 0x7fd0ec06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:14.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.007+0000 7fd119800640 1 --2- 172.21.15.143:0/3543001393 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd0ec06b2c0 0x7fd0ec06d780 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd104006fd0 tx=0x7fd104008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:14.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.011+0000 7fd111600640 1 -- 172.21.15.143:0/3543001393 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd11415b8a0 con 0x7fd11415e620 2024-01-16T15:01:15.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.251+0000 7fd11ac00640 1 -- 172.21.15.143:0/3543001393 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fd11415b8a0 con 0x7fd11415e620 2024-01-16T15:01:15.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.254+0000 7fd111600640 1 -- 172.21.15.143:0/3543001393 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fd11415b8a0 con 0x7fd11415e620 2024-01-16T15:01:15.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.257+0000 7fd11ac00640 1 -- 172.21.15.143:0/3543001393 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd0ec06b2c0 msgr2=0x7fd0ec06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:15.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.257+0000 7fd11ac00640 1 --2- 172.21.15.143:0/3543001393 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd0ec06b2c0 0x7fd0ec06d780 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd104006fd0 tx=0x7fd104008040 comp rx=0 tx=0).stop 2024-01-16T15:01:15.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.257+0000 7fd11ac00640 1 -- 172.21.15.143:0/3543001393 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd11415e620 msgr2=0x7fd11414e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:15.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.257+0000 7fd11ac00640 1 --2- 172.21.15.143:0/3543001393 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd11415e620 0x7fd11414e880 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd10c036040 tx=0x7fd10c004a40 comp rx=0 tx=0).stop 2024-01-16T15:01:15.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.257+0000 7fd11ac00640 1 -- 172.21.15.143:0/3543001393 shutdown_connections 2024-01-16T15:01:15.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.257+0000 7fd11ac00640 1 --2- 172.21.15.143:0/3543001393 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd0ec06b2c0 0x7fd0ec06d780 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:15.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.257+0000 7fd11ac00640 1 --2- 172.21.15.143:0/3543001393 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd11415e620 0x7fd11414e880 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:15.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.257+0000 7fd11ac00640 1 --2- 172.21.15.143:0/3543001393 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1140b0b50 0x7fd11414f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:15.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.257+0000 7fd11ac00640 1 --2- 172.21.15.143:0/3543001393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1140ae670 0x7fd114155400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:15.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.257+0000 7fd11ac00640 1 -- 172.21.15.143:0/3543001393 >> 172.21.15.143:0/3543001393 conn(0x7fd11406a500 msgr2=0x7fd1140ce2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:15.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.257+0000 7fd11ac00640 1 -- 172.21.15.143:0/3543001393 shutdown_connections 2024-01-16T15:01:15.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.257+0000 7fd11ac00640 1 -- 172.21.15.143:0/3543001393 wait complete. 2024-01-16T15:01:15.255 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T15:01:15.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.466+0000 7f1414400640 1 Processor -- start 2024-01-16T15:01:15.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.466+0000 7f1414400640 1 -- start start 2024-01-16T15:01:15.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.467+0000 7f1414400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f140c0cf450 0x7f140c0cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:15.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.467+0000 7f1414400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f140c0cdfb0 0x7f140c0ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:15.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.467+0000 7f1414400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f140c0cece0 con 0x7f140c15c7f0 2024-01-16T15:01:15.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.467+0000 7f1414400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f140c0cfe40 con 0x7f140c0cdfb0 2024-01-16T15:01:15.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.467+0000 7f1414400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f140c156010 con 0x7f140c0cf450 2024-01-16T15:01:15.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.468+0000 7f1412400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f140c0cdfb0 0x7f140c0ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:15.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.468+0000 7f1412400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f140c0cdfb0 0x7f140c0ce3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58980/0 (socket says 172.21.15.143:58980) 2024-01-16T15:01:15.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.468+0000 7f1412400640 1 -- 172.21.15.143:0/836640289 learned_addr learned my addr 172.21.15.143:0/836640289 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:15.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.468+0000 7f1412400640 1 -- 172.21.15.143:0/836640289 >> v2:172.21.15.204:3300/0 conn(0x7f140c0cf450 msgr2=0x7f140c0cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:15.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.468+0000 7f1412400640 1 --2- 172.21.15.143:0/836640289 >> v2:172.21.15.204:3300/0 conn(0x7f140c0cf450 0x7f140c0cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:15.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.468+0000 7f1412400640 1 -- 172.21.15.143:0/836640289 >> v1:172.21.15.143:6789/0 conn(0x7f140c15c7f0 legacy=0x7f140c06db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:15.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.468+0000 7f1412400640 1 -- 172.21.15.143:0/836640289 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f140c06d040 con 0x7f140c0cdfb0 2024-01-16T15:01:15.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.469+0000 7f1412400640 1 --2- 172.21.15.143:0/836640289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f140c0cdfb0 0x7f140c0ce3e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f140c070410 tx=0x7f140402bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d938180b7e884520 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:15.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.469+0000 7f140be00640 1 -- 172.21.15.143:0/836640289 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140402cb10 con 0x7f140c0cdfb0 2024-01-16T15:01:15.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.470+0000 7f140be00640 1 -- 172.21.15.143:0/836640289 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140402cc70 con 0x7f140c0cdfb0 2024-01-16T15:01:15.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.470+0000 7f140be00640 1 -- 172.21.15.143:0/836640289 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14040343f0 con 0x7f140c0cdfb0 2024-01-16T15:01:15.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.470+0000 7f1414400640 1 -- 172.21.15.143:0/836640289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f140c0cdfb0 msgr2=0x7f140c0ce3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:15.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.470+0000 7f1414400640 1 --2- 172.21.15.143:0/836640289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f140c0cdfb0 0x7f140c0ce3e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f140c070410 tx=0x7f140402bf70 comp rx=0 tx=0).stop 2024-01-16T15:01:15.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.470+0000 7f1414400640 1 -- 172.21.15.143:0/836640289 shutdown_connections 2024-01-16T15:01:15.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.470+0000 7f1414400640 1 --2- 172.21.15.143:0/836640289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f140c0cdfb0 0x7f140c0ce3e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:15.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.470+0000 7f1414400640 1 --2- 172.21.15.143:0/836640289 >> v2:172.21.15.204:3300/0 conn(0x7f140c0cf450 0x7f140c0cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:15.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.470+0000 7f1414400640 1 -- 172.21.15.143:0/836640289 >> 172.21.15.143:0/836640289 conn(0x7f140c06a920 msgr2=0x7f140c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:15.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.470+0000 7f1414400640 1 -- 172.21.15.143:0/836640289 shutdown_connections 2024-01-16T15:01:15.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.470+0000 7f1414400640 1 -- 172.21.15.143:0/836640289 wait complete. 2024-01-16T15:01:15.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.472+0000 7f1414400640 1 Processor -- start 2024-01-16T15:01:15.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.472+0000 7f1414400640 1 -- start start 2024-01-16T15:01:15.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.473+0000 7f1414400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f140c0cdfb0 0x7f140c149470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:15.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.473+0000 7f1414400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f140c0cf450 0x7f140c1571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:15.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.473+0000 7f1414400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f140c15c7f0 0x7f140c157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:15.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.473+0000 7f1414400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140c1499b0 con 0x7f140c0cf450 2024-01-16T15:01:15.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.473+0000 7f1414400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140c0acf50 con 0x7f140c15c7f0 2024-01-16T15:01:15.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.473+0000 7f1414400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f140c0ad230 con 0x7f140c0cdfb0 2024-01-16T15:01:15.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.473+0000 7f1411000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f140c0cf450 0x7f140c1571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:15.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.473+0000 7f1411000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f140c0cf450 0x7f140c1571d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58988/0 (socket says 172.21.15.143:58988) 2024-01-16T15:01:15.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.473+0000 7f1411000640 1 -- 172.21.15.143:0/1235098369 learned_addr learned my addr 172.21.15.143:0/1235098369 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:15.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.474+0000 7f1412400640 1 --2- 172.21.15.143:0/1235098369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f140c15c7f0 0x7f140c157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:15.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.474+0000 7f1411000640 1 -- 172.21.15.143:0/1235098369 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f140c0cdfb0 msgr2=0x7f140c149470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:15.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.474+0000 7f1411000640 1 --2- 172.21.15.143:0/1235098369 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f140c0cdfb0 0x7f140c149470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:15.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.474+0000 7f1411000640 1 -- 172.21.15.143:0/1235098369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f140c15c7f0 msgr2=0x7f140c157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:15.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.474+0000 7f1411000640 1 --2- 172.21.15.143:0/1235098369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f140c15c7f0 0x7f140c157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:15.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.474+0000 7f1411000640 1 -- 172.21.15.143:0/1235098369 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1404009b30 con 0x7f140c0cf450 2024-01-16T15:01:15.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.474+0000 7f1412400640 1 --2- 172.21.15.143:0/1235098369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f140c15c7f0 0x7f140c157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:01:15.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.474+0000 7f1411000640 1 --2- 172.21.15.143:0/1235098369 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f140c0cf450 0x7f140c1571d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f13f400b4a0 tx=0x7f13f400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:15.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.474+0000 7f1409600640 1 -- 172.21.15.143:0/1235098369 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f4010610 con 0x7f140c0cf450 2024-01-16T15:01:15.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.474+0000 7f1409600640 1 -- 172.21.15.143:0/1235098369 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f4010770 con 0x7f140c0cf450 2024-01-16T15:01:15.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.474+0000 7f1409600640 1 -- 172.21.15.143:0/1235098369 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f4010b20 con 0x7f140c0cf450 2024-01-16T15:01:15.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.474+0000 7f1414400640 1 -- 172.21.15.143:0/1235098369 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f140c0ad510 con 0x7f140c0cf450 2024-01-16T15:01:15.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.474+0000 7f1414400640 1 -- 172.21.15.143:0/1235098369 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f140c0ac220 con 0x7f140c0cf450 2024-01-16T15:01:15.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.475+0000 7f1414400640 1 -- 172.21.15.143:0/1235098369 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f140c067470 con 0x7f140c0cf450 2024-01-16T15:01:15.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.476+0000 7f1409600640 1 -- 172.21.15.143:0/1235098369 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f13f4010c80 con 0x7f140c0cf450 2024-01-16T15:01:15.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.477+0000 7f1409600640 1 --2- 172.21.15.143:0/1235098369 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f13e406b2c0 0x7f13e406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:15.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.477+0000 7f1409600640 1 -- 172.21.15.143:0/1235098369 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f13f4089500 con 0x7f140c0cf450 2024-01-16T15:01:15.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.478+0000 7f1411a00640 1 --2- 172.21.15.143:0/1235098369 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f13e406b2c0 0x7f13e406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:15.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.478+0000 7f1411a00640 1 --2- 172.21.15.143:0/1235098369 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f13e406b2c0 0x7f13e406d780 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f13fc000f60 tx=0x7f13fc0079b0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:15.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.482+0000 7f1409600640 1 -- 172.21.15.143:0/1235098369 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f13f40546b0 con 0x7f140c0cf450 2024-01-16T15:01:15.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.668+0000 7f1414400640 1 -- 172.21.15.143:0/1235098369 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f140c0cd730 con 0x7f140c0cf450 2024-01-16T15:01:15.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.670+0000 7f1409600640 1 -- 172.21.15.143:0/1235098369 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+12917 (secure 0 0 0) 0x7f13f4014090 con 0x7f140c0cf450 2024-01-16T15:01:15.650 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:01:15.650 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":30,"fsid":"c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0","created":"2024-01-16T14:59:38.532601+0000","modified":"2024-01-16T15:01:11.789063+0000","last_up_change":"2024-01-16T14:59:50.983699+0000","last_in_change":"2024-01-16T14:59:42.056297+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:59:51.727720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f4c8891d-ab29-472a-907d-8446f20f7012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6801","nonce":1569293626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6805","nonce":1569293626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6814","nonce":1569293626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6809","nonce":1569293626}]},"public_addr":"172.21.15.143:6801/1569293626","cluster_addr":"172.21.15.143:6805/1569293626","heartbeat_back_addr":"172.21.15.143:6814/1569293626","heartbeat_front_addr":"172.21.15.143:6809/1569293626","state":["exists","up"]},{"osd":1,"uuid":"c4a2788d-8b89-4992-82b2-7f4b9debd509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6804","nonce":1276841168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6808","nonce":1276841168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6813","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6815","nonce":1276841168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6811","nonce":1276841168}]},"public_addr":"172.21.15.143:6804/1276841168","cluster_addr":"172.21.15.143:6808/1276841168","heartbeat_back_addr":"172.21.15.143:6815/1276841168","heartbeat_front_addr":"172.21.15.143:6811/1276841168","state":["exists","up"]},{"osd":2,"uuid":"64645c1d-31b2-4794-b0f2-7ccf8df30393","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6817","nonce":1217670170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6819","nonce":1217670170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6823","nonce":1217670170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6821","nonce":1217670170}]},"public_addr":"172.21.15.143:6817/1217670170","cluster_addr":"172.21.15.143:6819/1217670170","heartbeat_back_addr":"172.21.15.143:6823/1217670170","heartbeat_front_addr":"172.21.15.143:6821/1217670170","state":["exists","up"]},{"osd":3,"uuid":"73674129-a84e-4d31-8247-7d0179ae6df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6825","nonce":3414188012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6827","nonce":3414188012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6831","nonce":3414188012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6829","nonce":3414188012}]},"public_addr":"172.21.15.143:6825/3414188012","cluster_addr":"172.21.15.143:6827/3414188012","heartbeat_back_addr":"172.21.15.143:6831/3414188012","heartbeat_front_addr":"172.21.15.143:6829/3414188012","state":["exists","up"]},{"osd":4,"uuid":"cea28c9b-2f1c-4087-b415-9960789131d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6811","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6812","nonce":1070255716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6816","nonce":1070255716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6823","nonce":1070255716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6820","nonce":1070255716}]},"public_addr":"172.21.15.204:6812/1070255716","cluster_addr":"172.21.15.204:6816/1070255716","heartbeat_back_addr":"172.21.15.204:6823/1070255716","heartbeat_front_addr":"172.21.15.204:6820/1070255716","state":["exists","up"]},{"osd":5,"uuid":"33b63560-5574-4e78-983f-b398d67a2cb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6809","nonce":3605417837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6813","nonce":3605417837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6819","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6821","nonce":3605417837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6817","nonce":3605417837}]},"public_addr":"172.21.15.204:6809/3605417837","cluster_addr":"172.21.15.204:6813/3605417837","heartbeat_back_addr":"172.21.15.204:6821/3605417837","heartbeat_front_addr":"172.21.15.204:6817/3605417837","state":["exists","up"]},{"osd":6,"uuid":"df397427-8940-4eee-9c79-eece669d735d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6801","nonce":1085000534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6803","nonce":1085000534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6808","nonce":1085000534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6805","nonce":1085000534}]},"public_addr":"172.21.15.204:6801/1085000534","cluster_addr":"172.21.15.204:6803/1085000534","heartbeat_back_addr":"172.21.15.204:6808/1085000534","heartbeat_front_addr":"172.21.15.204:6805/1085000534","state":["exists","up"]},{"osd":7,"uuid":"a2e5a15c-8312-405a-ac02-f7e1fc9df2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6825","nonce":1337220609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6827","nonce":1337220609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6831","nonce":1337220609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6829","nonce":1337220609}]},"public_addr":"172.21.15.204:6825/1337220609","cluster_addr":"172.21.15.204:6827/1337220609","heartbeat_back_addr":"172.21.15.204:6831/1337220609","heartbeat_front_addr":"172.21.15.204:6829/1337220609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:49.984507+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.395306+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.632998+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.373627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.342920+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.438366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.331926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.406657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T15:01:15.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.673+0000 7f1414400640 1 -- 172.21.15.143:0/1235098369 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f13e406b2c0 msgr2=0x7f13e406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:15.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.673+0000 7f1414400640 1 --2- 172.21.15.143:0/1235098369 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f13e406b2c0 0x7f13e406d780 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f13fc000f60 tx=0x7f13fc0079b0 comp rx=0 tx=0).stop 2024-01-16T15:01:15.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.673+0000 7f1414400640 1 -- 172.21.15.143:0/1235098369 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f140c0cf450 msgr2=0x7f140c1571d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:15.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.673+0000 7f1414400640 1 --2- 172.21.15.143:0/1235098369 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f140c0cf450 0x7f140c1571d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f13f400b4a0 tx=0x7f13f400b960 comp rx=0 tx=0).stop 2024-01-16T15:01:15.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.673+0000 7f1414400640 1 -- 172.21.15.143:0/1235098369 shutdown_connections 2024-01-16T15:01:15.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.673+0000 7f1414400640 1 --2- 172.21.15.143:0/1235098369 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f13e406b2c0 0x7f13e406d780 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:15.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.673+0000 7f1414400640 1 --2- 172.21.15.143:0/1235098369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f140c15c7f0 0x7f140c157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:15.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.673+0000 7f1414400640 1 --2- 172.21.15.143:0/1235098369 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f140c0cf450 0x7f140c1571d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:15.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.673+0000 7f1414400640 1 --2- 172.21.15.143:0/1235098369 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f140c0cdfb0 0x7f140c149470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:15.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.673+0000 7f1414400640 1 -- 172.21.15.143:0/1235098369 >> 172.21.15.143:0/1235098369 conn(0x7f140c06a920 msgr2=0x7f140c06c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:15.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.673+0000 7f1414400640 1 -- 172.21.15.143:0/1235098369 shutdown_connections 2024-01-16T15:01:15.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.673+0000 7f1414400640 1 -- 172.21.15.143:0/1235098369 wait complete. 2024-01-16T15:01:15.670 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-16T14:59:51.727720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-16T15:01:15.671 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-16T15:01:15.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.896+0000 7fb062400640 1 Processor -- start 2024-01-16T15:01:15.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.896+0000 7fb062400640 1 -- start start 2024-01-16T15:01:15.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.897+0000 7fb062400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb05c0b0b40 0x7fb05c0b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:15.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.897+0000 7fb062400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb05c15ded0 0x7fb05c15e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:15.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.897+0000 7fb062400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb05c06d430 con 0x7fb05c15a1c0 2024-01-16T15:01:15.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.897+0000 7fb062400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb05c15ec50 con 0x7fb05c0b0b40 2024-01-16T15:01:15.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.897+0000 7fb062400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb05c06cb00 con 0x7fb05c15ded0 2024-01-16T15:01:15.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.898+0000 7fb05be00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb05c0b0b40 0x7fb05c0b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:15.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.898+0000 7fb061a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb05c15ded0 0x7fb05c15e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:15.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.898+0000 7fb05be00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb05c0b0b40 0x7fb05c0b0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:51468/0 (socket says 172.21.15.143:51468) 2024-01-16T15:01:15.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.898+0000 7fb061a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb05c15ded0 0x7fb05c15e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:43132/0 (socket says 172.21.15.143:43132) 2024-01-16T15:01:15.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.898+0000 7fb061000640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fb05c15a1c0 0x7fb05c06d8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:32858/0 (socket says 172.21.15.143:32858) 2024-01-16T15:01:15.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.898+0000 7fb05be00640 1 -- 172.21.15.143:0/2400085796 learned_addr learned my addr 172.21.15.143:0/2400085796 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:15.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.899+0000 7fb05b400640 1 -- 172.21.15.143:0/2400085796 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1675591542 0 0) 0x7fb05c06d430 con 0x7fb05c15a1c0 2024-01-16T15:01:15.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.899+0000 7fb05b400640 1 -- 172.21.15.143:0/2400085796 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb03c003520 con 0x7fb05c15a1c0 2024-01-16T15:01:15.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.899+0000 7fb061a00640 1 -- 172.21.15.143:0/2400085796 >> v2:172.21.15.204:3300/0 conn(0x7fb05c0b0b40 msgr2=0x7fb05c0b0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:15.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.899+0000 7fb061a00640 1 --2- 172.21.15.143:0/2400085796 >> v2:172.21.15.204:3300/0 conn(0x7fb05c0b0b40 0x7fb05c0b0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:15.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.899+0000 7fb061a00640 1 -- 172.21.15.143:0/2400085796 >> v1:172.21.15.204:6789/0 conn(0x7fb05c15a1c0 legacy=0x7fb05c06d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:15.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.899+0000 7fb061a00640 1 -- 172.21.15.143:0/2400085796 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb05c0b14e0 con 0x7fb05c15ded0 2024-01-16T15:01:15.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.901+0000 7fb061a00640 1 --2- 172.21.15.143:0/2400085796 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb05c15ded0 0x7fb05c15e360 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb054009d90 tx=0x7fb05402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71650cb57cb84986 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:15.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.901+0000 7fb05b400640 1 -- 172.21.15.143:0/2400085796 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05402cc50 con 0x7fb05c15ded0 2024-01-16T15:01:15.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.901+0000 7fb05b400640 1 -- 172.21.15.143:0/2400085796 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb05402cdb0 con 0x7fb05c15ded0 2024-01-16T15:01:15.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.901+0000 7fb05b400640 1 -- 172.21.15.143:0/2400085796 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb054034660 con 0x7fb05c15ded0 2024-01-16T15:01:15.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.902+0000 7fb062400640 1 -- 172.21.15.143:0/2400085796 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb05c15ded0 msgr2=0x7fb05c15e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:15.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.902+0000 7fb062400640 1 --2- 172.21.15.143:0/2400085796 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb05c15ded0 0x7fb05c15e360 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb054009d90 tx=0x7fb05402bea0 comp rx=0 tx=0).stop 2024-01-16T15:01:15.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.902+0000 7fb062400640 1 -- 172.21.15.143:0/2400085796 shutdown_connections 2024-01-16T15:01:15.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.902+0000 7fb062400640 1 --2- 172.21.15.143:0/2400085796 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb05c15ded0 0x7fb05c15e360 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:15.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.902+0000 7fb062400640 1 --2- 172.21.15.143:0/2400085796 >> v2:172.21.15.204:3300/0 conn(0x7fb05c0b0b40 0x7fb05c0b0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:15.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.902+0000 7fb062400640 1 -- 172.21.15.143:0/2400085796 >> 172.21.15.143:0/2400085796 conn(0x7fb05c0cbb40 msgr2=0x7fb05c0cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:15.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.902+0000 7fb062400640 1 -- 172.21.15.143:0/2400085796 shutdown_connections 2024-01-16T15:01:15.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.902+0000 7fb062400640 1 -- 172.21.15.143:0/2400085796 wait complete. 2024-01-16T15:01:15.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.904+0000 7fb062400640 1 Processor -- start 2024-01-16T15:01:15.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.904+0000 7fb062400640 1 -- start start 2024-01-16T15:01:15.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.905+0000 7fb062400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb05c0b0b40 0x7fb05c15a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:15.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.905+0000 7fb062400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb05c15a1c0 0x7fb05c15aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:15.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.905+0000 7fb062400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb05c15ded0 0x7fb05c15c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:15.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.905+0000 7fb062400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05c0c4000 con 0x7fb05c15a1c0 2024-01-16T15:01:15.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.905+0000 7fb062400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05c0c3430 con 0x7fb05c0b0b40 2024-01-16T15:01:15.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.905+0000 7fb062400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb05c0c36d0 con 0x7fb05c15ded0 2024-01-16T15:01:15.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.905+0000 7fb05be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb05c15a1c0 0x7fb05c15aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:15.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.905+0000 7fb05be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb05c15a1c0 0x7fb05c15aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59002/0 (socket says 172.21.15.143:59002) 2024-01-16T15:01:15.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.905+0000 7fb05be00640 1 -- 172.21.15.143:0/3197185334 learned_addr learned my addr 172.21.15.143:0/3197185334 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:15.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.906+0000 7fb061a00640 1 --2- 172.21.15.143:0/3197185334 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb05c15ded0 0x7fb05c15c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:15.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.906+0000 7fb061000640 1 --2- 172.21.15.143:0/3197185334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb05c0b0b40 0x7fb05c15a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:15.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.906+0000 7fb05be00640 1 -- 172.21.15.143:0/3197185334 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb05c15ded0 msgr2=0x7fb05c15c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:15.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.906+0000 7fb05be00640 1 --2- 172.21.15.143:0/3197185334 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb05c15ded0 0x7fb05c15c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:15.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.906+0000 7fb05be00640 1 -- 172.21.15.143:0/3197185334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb05c0b0b40 msgr2=0x7fb05c15a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:15.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.906+0000 7fb05be00640 1 --2- 172.21.15.143:0/3197185334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb05c0b0b40 0x7fb05c15a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:15.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.906+0000 7fb05be00640 1 -- 172.21.15.143:0/3197185334 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb054009a70 con 0x7fb05c15a1c0 2024-01-16T15:01:15.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.906+0000 7fb061a00640 1 --2- 172.21.15.143:0/3197185334 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb05c15ded0 0x7fb05c15c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:01:15.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.906+0000 7fb05be00640 1 --2- 172.21.15.143:0/3197185334 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb05c15a1c0 0x7fb05c15aae0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb044009fd0 tx=0x7fb04400ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:15.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.906+0000 7fb058c00640 1 -- 172.21.15.143:0/3197185334 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04400c940 con 0x7fb05c15a1c0 2024-01-16T15:01:15.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.906+0000 7fb058c00640 1 -- 172.21.15.143:0/3197185334 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb04400caa0 con 0x7fb05c15a1c0 2024-01-16T15:01:15.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.906+0000 7fb058c00640 1 -- 172.21.15.143:0/3197185334 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04400ce30 con 0x7fb05c15a1c0 2024-01-16T15:01:15.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.906+0000 7fb062400640 1 -- 172.21.15.143:0/3197185334 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb05c15ea20 con 0x7fb05c15a1c0 2024-01-16T15:01:15.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.907+0000 7fb062400640 1 -- 172.21.15.143:0/3197185334 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb05c0c3c90 con 0x7fb05c15a1c0 2024-01-16T15:01:15.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.908+0000 7fb062400640 1 -- 172.21.15.143:0/3197185334 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb05c0c41b0 con 0x7fb05c15a1c0 2024-01-16T15:01:15.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.909+0000 7fb058c00640 1 -- 172.21.15.143:0/3197185334 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fb044013470 con 0x7fb05c15a1c0 2024-01-16T15:01:15.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.909+0000 7fb058c00640 1 --2- 172.21.15.143:0/3197185334 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb03406b2c0 0x7fb03406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:15.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.909+0000 7fb058c00640 1 -- 172.21.15.143:0/3197185334 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb044089d70 con 0x7fb05c15a1c0 2024-01-16T15:01:15.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.910+0000 7fb061000640 1 --2- 172.21.15.143:0/3197185334 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb03406b2c0 0x7fb03406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:15.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.910+0000 7fb061000640 1 --2- 172.21.15.143:0/3197185334 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb03406b2c0 0x7fb03406d780 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb04c0010e0 tx=0x7fb04c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:15.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:15.914+0000 7fb058c00640 1 -- 172.21.15.143:0/3197185334 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb05c0c41b0 con 0x7fb05c15a1c0 2024-01-16T15:01:16.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.100+0000 7fb062400640 1 -- 172.21.15.143:0/3197185334 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb05c09cd00 con 0x7fb05c15a1c0 2024-01-16T15:01:16.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.101+0000 7fb058c00640 1 -- 172.21.15.143:0/3197185334 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb0440170a0 con 0x7fb05c15a1c0 2024-01-16T15:01:16.081 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-16T15:01:16.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.104+0000 7fb062400640 1 -- 172.21.15.143:0/3197185334 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb03406b2c0 msgr2=0x7fb03406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:16.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.104+0000 7fb062400640 1 --2- 172.21.15.143:0/3197185334 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb03406b2c0 0x7fb03406d780 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb04c0010e0 tx=0x7fb04c009040 comp rx=0 tx=0).stop 2024-01-16T15:01:16.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.104+0000 7fb062400640 1 -- 172.21.15.143:0/3197185334 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb05c15a1c0 msgr2=0x7fb05c15aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:16.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.104+0000 7fb062400640 1 --2- 172.21.15.143:0/3197185334 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb05c15a1c0 0x7fb05c15aae0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb044009fd0 tx=0x7fb04400ed40 comp rx=0 tx=0).stop 2024-01-16T15:01:16.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.104+0000 7fb062400640 1 -- 172.21.15.143:0/3197185334 shutdown_connections 2024-01-16T15:01:16.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.104+0000 7fb062400640 1 --2- 172.21.15.143:0/3197185334 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb03406b2c0 0x7fb03406d780 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:16.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.104+0000 7fb062400640 1 --2- 172.21.15.143:0/3197185334 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb05c15ded0 0x7fb05c15c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:16.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.104+0000 7fb062400640 1 --2- 172.21.15.143:0/3197185334 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb05c15a1c0 0x7fb05c15aae0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:16.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.104+0000 7fb062400640 1 --2- 172.21.15.143:0/3197185334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb05c0b0b40 0x7fb05c15a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:16.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.104+0000 7fb062400640 1 -- 172.21.15.143:0/3197185334 >> 172.21.15.143:0/3197185334 conn(0x7fb05c0cbb40 msgr2=0x7fb05c09c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:16.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.104+0000 7fb062400640 1 -- 172.21.15.143:0/3197185334 shutdown_connections 2024-01-16T15:01:16.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.104+0000 7fb062400640 1 -- 172.21.15.143:0/3197185334 wait complete. 2024-01-16T15:01:16.103 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-16T15:01:16.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.329+0000 7f9188800640 1 Processor -- start 2024-01-16T15:01:16.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.330+0000 7f9188800640 1 -- start start 2024-01-16T15:01:16.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.331+0000 7f9188800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f918015d3f0 0x7f918006d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:16.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.331+0000 7f9188800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91800ac840 0x7f91800b5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:16.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.331+0000 7f9188800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91800b4170 con 0x7f918006d580 2024-01-16T15:01:16.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.331+0000 7f9188800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91800b4450 con 0x7f918015d3f0 2024-01-16T15:01:16.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.331+0000 7f9188800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91800b4730 con 0x7f91800ac840 2024-01-16T15:01:16.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.331+0000 7f9185e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f918015d3f0 0x7f918006d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:16.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.332+0000 7f9185e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f918015d3f0 0x7f918006d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59004/0 (socket says 172.21.15.143:59004) 2024-01-16T15:01:16.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.332+0000 7f9185e00640 1 -- 172.21.15.143:0/342263383 learned_addr learned my addr 172.21.15.143:0/342263383 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:16.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.332+0000 7f9185400640 1 --2- 172.21.15.143:0/342263383 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91800ac840 0x7f91800b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:16.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.332+0000 7f9185e00640 1 -- 172.21.15.143:0/342263383 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91800ac840 msgr2=0x7f91800b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:16.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.332+0000 7f9185e00640 1 --2- 172.21.15.143:0/342263383 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91800ac840 0x7f91800b5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:16.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.332+0000 7f9185e00640 1 -- 172.21.15.143:0/342263383 >> v1:172.21.15.204:6789/0 conn(0x7f918006d580 legacy=0x7f91800ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:16.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.332+0000 7f9185e00640 1 -- 172.21.15.143:0/342263383 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91800b4a10 con 0x7f918015d3f0 2024-01-16T15:01:16.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.332+0000 7f9185400640 1 --2- 172.21.15.143:0/342263383 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91800ac840 0x7f91800b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:01:16.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.333+0000 7f9185e00640 1 --2- 172.21.15.143:0/342263383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f918015d3f0 0x7f918006d040 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9170009870 tx=0x7f917002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d74ee03d998fb79e server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:16.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.333+0000 7f9184a00640 1 -- 172.21.15.143:0/342263383 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9170031640 con 0x7f918015d3f0 2024-01-16T15:01:16.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.334+0000 7f9184a00640 1 -- 172.21.15.143:0/342263383 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91700317a0 con 0x7f918015d3f0 2024-01-16T15:01:16.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.334+0000 7f9184a00640 1 -- 172.21.15.143:0/342263383 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9170031b30 con 0x7f918015d3f0 2024-01-16T15:01:16.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.334+0000 7f9188800640 1 -- 172.21.15.143:0/342263383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f918015d3f0 msgr2=0x7f918006d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:16.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.334+0000 7f9188800640 1 --2- 172.21.15.143:0/342263383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f918015d3f0 0x7f918006d040 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9170009870 tx=0x7f917002b8f0 comp rx=0 tx=0).stop 2024-01-16T15:01:16.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.334+0000 7f9188800640 1 -- 172.21.15.143:0/342263383 shutdown_connections 2024-01-16T15:01:16.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.334+0000 7f9188800640 1 --2- 172.21.15.143:0/342263383 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91800ac840 0x7f91800b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:16.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.334+0000 7f9188800640 1 --2- 172.21.15.143:0/342263383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f918015d3f0 0x7f918006d040 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:16.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.334+0000 7f9188800640 1 -- 172.21.15.143:0/342263383 >> 172.21.15.143:0/342263383 conn(0x7f9180071d40 msgr2=0x7f918006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:16.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.334+0000 7f9188800640 1 -- 172.21.15.143:0/342263383 shutdown_connections 2024-01-16T15:01:16.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.334+0000 7f9188800640 1 -- 172.21.15.143:0/342263383 wait complete. 2024-01-16T15:01:16.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.336+0000 7f9188800640 1 Processor -- start 2024-01-16T15:01:16.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.336+0000 7f9188800640 1 -- start start 2024-01-16T15:01:16.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.337+0000 7f9188800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f918006d580 0x7f918015c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:16.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.337+0000 7f9188800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91800ac840 0x7f9180159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:16.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.337+0000 7f9188800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f918015d3f0 0x7f918013e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:16.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.337+0000 7f9188800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918015a110 con 0x7f91800ac840 2024-01-16T15:01:16.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.337+0000 7f9188800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918013ee80 con 0x7f918006d580 2024-01-16T15:01:16.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.337+0000 7f9188800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f918013f170 con 0x7f918015d3f0 2024-01-16T15:01:16.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.337+0000 7f9185400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91800ac840 0x7f9180159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:16.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.337+0000 7f9185400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91800ac840 0x7f9180159bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59006/0 (socket says 172.21.15.143:59006) 2024-01-16T15:01:16.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.337+0000 7f9185400640 1 -- 172.21.15.143:0/2215498197 learned_addr learned my addr 172.21.15.143:0/2215498197 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:16.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.338+0000 7f9185e00640 1 --2- 172.21.15.143:0/2215498197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f918006d580 0x7f918015c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:16.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.338+0000 7f9186800640 1 --2- 172.21.15.143:0/2215498197 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f918015d3f0 0x7f918013e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:16.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.338+0000 7f9185400640 1 -- 172.21.15.143:0/2215498197 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f918015d3f0 msgr2=0x7f918013e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:16.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.338+0000 7f9185400640 1 --2- 172.21.15.143:0/2215498197 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f918015d3f0 0x7f918013e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:16.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.338+0000 7f9185400640 1 -- 172.21.15.143:0/2215498197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f918006d580 msgr2=0x7f918015c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:16.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.338+0000 7f9185400640 1 --2- 172.21.15.143:0/2215498197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f918006d580 0x7f918015c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:16.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.338+0000 7f9185400640 1 -- 172.21.15.143:0/2215498197 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91700094d0 con 0x7f91800ac840 2024-01-16T15:01:16.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.338+0000 7f9185400640 1 --2- 172.21.15.143:0/2215498197 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91800ac840 0x7f9180159bd0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f916800c7f0 tx=0x7f916800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:16.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.338+0000 7f917e000640 1 -- 172.21.15.143:0/2215498197 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916800cea0 con 0x7f91800ac840 2024-01-16T15:01:16.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.338+0000 7f917e000640 1 -- 172.21.15.143:0/2215498197 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9168007a30 con 0x7f91800ac840 2024-01-16T15:01:16.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.338+0000 7f917e000640 1 -- 172.21.15.143:0/2215498197 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9168007dc0 con 0x7f91800ac840 2024-01-16T15:01:16.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.339+0000 7f9188800640 1 -- 172.21.15.143:0/2215498197 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f918015cb30 con 0x7f91800ac840 2024-01-16T15:01:16.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.339+0000 7f9188800640 1 -- 172.21.15.143:0/2215498197 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f918013f710 con 0x7f91800ac840 2024-01-16T15:01:16.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.340+0000 7f9188800640 1 -- 172.21.15.143:0/2215498197 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9180067470 con 0x7f91800ac840 2024-01-16T15:01:16.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.341+0000 7f917e000640 1 -- 172.21.15.143:0/2215498197 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f91680113f0 con 0x7f91800ac840 2024-01-16T15:01:16.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.341+0000 7f917e000640 1 --2- 172.21.15.143:0/2215498197 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f915806b2c0 0x7f915806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:16.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.342+0000 7f917e000640 1 -- 172.21.15.143:0/2215498197 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9168089a50 con 0x7f91800ac840 2024-01-16T15:01:16.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.342+0000 7f9185e00640 1 --2- 172.21.15.143:0/2215498197 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f915806b2c0 0x7f915806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:16.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.343+0000 7f9185e00640 1 --2- 172.21.15.143:0/2215498197 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f915806b2c0 0x7f915806d780 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9170036000 tx=0x7f917003c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:16.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.346+0000 7f917e000640 1 -- 172.21.15.143:0/2215498197 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9168054b70 con 0x7f91800ac840 2024-01-16T15:01:16.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.540+0000 7f9188800640 1 -- 172.21.15.143:0/2215498197 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f91800b3330 con 0x7f91800ac840 2024-01-16T15:01:16.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.541+0000 7f917e000640 1 -- 172.21.15.143:0/2215498197 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) v1 ==== 84+0+65 (secure 0 0 0) 0x7f916801f090 con 0x7f91800ac840 2024-01-16T15:01:16.521 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:01:16.521 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-16T15:01:16.521 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-16T15:01:16.521 INFO:teuthology.orchestra.run.smithi143.stdout: "poolnum": 1, 2024-01-16T15:01:16.521 INFO:teuthology.orchestra.run.smithi143.stdout: "poolname": ".mgr" 2024-01-16T15:01:16.521 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:01:16.521 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-16T15:01:16.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.544+0000 7f9188800640 1 -- 172.21.15.143:0/2215498197 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f915806b2c0 msgr2=0x7f915806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:16.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.544+0000 7f9188800640 1 --2- 172.21.15.143:0/2215498197 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f915806b2c0 0x7f915806d780 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9170036000 tx=0x7f917003c040 comp rx=0 tx=0).stop 2024-01-16T15:01:16.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.544+0000 7f9188800640 1 -- 172.21.15.143:0/2215498197 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91800ac840 msgr2=0x7f9180159bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:16.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.544+0000 7f9188800640 1 --2- 172.21.15.143:0/2215498197 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91800ac840 0x7f9180159bd0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f916800c7f0 tx=0x7f916800ccb0 comp rx=0 tx=0).stop 2024-01-16T15:01:16.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.544+0000 7f9188800640 1 -- 172.21.15.143:0/2215498197 shutdown_connections 2024-01-16T15:01:16.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.544+0000 7f9188800640 1 --2- 172.21.15.143:0/2215498197 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f915806b2c0 0x7f915806d780 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:16.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.544+0000 7f9188800640 1 --2- 172.21.15.143:0/2215498197 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f918015d3f0 0x7f918013e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:16.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.544+0000 7f9188800640 1 --2- 172.21.15.143:0/2215498197 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91800ac840 0x7f9180159bd0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:16.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.544+0000 7f9188800640 1 --2- 172.21.15.143:0/2215498197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f918006d580 0x7f918015c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:16.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.544+0000 7f9188800640 1 -- 172.21.15.143:0/2215498197 >> 172.21.15.143:0/2215498197 conn(0x7f9180071d40 msgr2=0x7f91801537b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:16.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.544+0000 7f9188800640 1 -- 172.21.15.143:0/2215498197 shutdown_connections 2024-01-16T15:01:16.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.544+0000 7f9188800640 1 -- 172.21.15.143:0/2215498197 wait complete. 2024-01-16T15:01:16.543 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-16T15:01:16.543 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-16T15:01:16.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.763+0000 7f448de00640 1 Processor -- start 2024-01-16T15:01:16.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.763+0000 7f448de00640 1 -- start start 2024-01-16T15:01:16.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.765+0000 7f448de00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44880ce6c0 0x7f44880cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:16.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.765+0000 7f448de00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44880cece0 con 0x7f448815c7f0 2024-01-16T15:01:16.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.765+0000 7f448de00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f448806d040 con 0x7f44880cf450 2024-01-16T15:01:16.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.765+0000 7f448de00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44880cfe40 con 0x7f44880ce6c0 2024-01-16T15:01:16.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.765+0000 7f4486a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f448815c7f0 0x7f448806db20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57750/0 (socket says 172.21.15.143:57750) 2024-01-16T15:01:16.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.765+0000 7f4486a00640 1 -- 172.21.15.143:0/1382608873 learned_addr learned my addr 172.21.15.143:0/1382608873 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:01:16.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.766+0000 7f4487400640 1 --2- 172.21.15.143:0/1382608873 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44880ce6c0 0x7f44880cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:16.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.766+0000 7f4486000640 1 -- 172.21.15.143:0/1382608873 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 423016189 0 0) 0x7f44880cece0 con 0x7f448815c7f0 2024-01-16T15:01:16.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.766+0000 7f4486000640 1 -- 172.21.15.143:0/1382608873 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4468003520 con 0x7f448815c7f0 2024-01-16T15:01:16.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.766+0000 7f4486000640 1 -- 172.21.15.143:0/1382608873 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3909015368 0 0) 0x7f4468003520 con 0x7f448815c7f0 2024-01-16T15:01:16.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.766+0000 7f4486000640 1 -- 172.21.15.143:0/1382608873 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44880ce6c0 msgr2=0x7f44880cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:16.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.766+0000 7f4486000640 1 --2- 172.21.15.143:0/1382608873 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44880ce6c0 0x7f44880cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:16.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.766+0000 7f4486000640 1 -- 172.21.15.143:0/1382608873 >> v1:172.21.15.204:6789/0 conn(0x7f44880cf450 legacy=0x7f44880cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:16.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.766+0000 7f4486000640 1 -- 172.21.15.143:0/1382608873 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f448806d320 con 0x7f448815c7f0 2024-01-16T15:01:16.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.766+0000 7f4486000640 1 -- 172.21.15.143:0/1382608873 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7f4478001670 con 0x7f448815c7f0 2024-01-16T15:01:16.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.766+0000 7f4486000640 1 -- 172.21.15.143:0/1382608873 >> v1:172.21.15.143:6789/0 conn(0x7f448815c7f0 legacy=0x7f448806db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:16.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.767+0000 7f4486000640 1 --2- 172.21.15.143:0/1382608873 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44880ce6c0 0x7f44680039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:16.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.767+0000 7f4486000640 1 --2- 172.21.15.143:0/1382608873 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4468003ee0 0x7f44680047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:16.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.767+0000 7f4486000640 1 --2- 172.21.15.143:0/1382608873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4468004d00 0x7f4468005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:16.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.767+0000 7f4486000640 1 -- 172.21.15.143:0/1382608873 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44880cfe40 con 0x7f4468003ee0 2024-01-16T15:01:16.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.767+0000 7f4486000640 1 -- 172.21.15.143:0/1382608873 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446800b9e0 con 0x7f4468004d00 2024-01-16T15:01:16.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.767+0000 7f4486000640 1 -- 172.21.15.143:0/1382608873 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f446800bba0 con 0x7f44880ce6c0 2024-01-16T15:01:16.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.767+0000 7f447de00640 1 --2- 172.21.15.143:0/1382608873 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4468003ee0 0x7f44680047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:16.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.767+0000 7f447de00640 1 -- 172.21.15.143:0/1382608873 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44880ce6c0 msgr2=0x7f44680039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:16.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.767+0000 7f447de00640 1 --2- 172.21.15.143:0/1382608873 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44880ce6c0 0x7f44680039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:16.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.767+0000 7f447de00640 1 -- 172.21.15.143:0/1382608873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4468004d00 msgr2=0x7f4468005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:16.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.767+0000 7f4487400640 1 --2- 172.21.15.143:0/1382608873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4468004d00 0x7f4468005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:16.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.767+0000 7f447de00640 1 --2- 172.21.15.143:0/1382608873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4468004d00 0x7f4468005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:16.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.767+0000 7f447de00640 1 -- 172.21.15.143:0/1382608873 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4468003520 con 0x7f4468003ee0 2024-01-16T15:01:16.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.767+0000 7f4487400640 1 --2- 172.21.15.143:0/1382608873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4468004d00 0x7f4468005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:01:16.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.768+0000 7f447de00640 1 --2- 172.21.15.143:0/1382608873 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4468003ee0 0x7f44680047c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4470001910 tx=0x7f44680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b1bc001896acae server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:16.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.769+0000 7f4486000640 1 -- 172.21.15.143:0/1382608873 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470032600 con 0x7f4468003ee0 2024-01-16T15:01:16.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.769+0000 7f4486000640 1 -- 172.21.15.143:0/1382608873 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447002e050 con 0x7f4468003ee0 2024-01-16T15:01:16.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.769+0000 7f4486000640 1 -- 172.21.15.143:0/1382608873 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44700329c0 con 0x7f4468003ee0 2024-01-16T15:01:16.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.769+0000 7f448de00640 1 -- 172.21.15.143:0/1382608873 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4468003ee0 msgr2=0x7f44680047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:16.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.769+0000 7f448de00640 1 --2- 172.21.15.143:0/1382608873 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4468003ee0 0x7f44680047c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4470001910 tx=0x7f44680057f0 comp rx=0 tx=0).stop 2024-01-16T15:01:16.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.769+0000 7f4487400640 1 -- 172.21.15.143:0/1382608873 reap_dead start 2024-01-16T15:01:16.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.769+0000 7f448de00640 1 -- 172.21.15.143:0/1382608873 shutdown_connections 2024-01-16T15:01:16.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.769+0000 7f448de00640 1 -- 172.21.15.143:0/1382608873 >> 172.21.15.143:0/1382608873 conn(0x7f448806a920 msgr2=0x7f448806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:16.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.769+0000 7f448de00640 1 -- 172.21.15.143:0/1382608873 shutdown_connections 2024-01-16T15:01:16.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.769+0000 7f448de00640 1 -- 172.21.15.143:0/1382608873 wait complete. 2024-01-16T15:01:16.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.771+0000 7f448de00640 1 Processor -- start 2024-01-16T15:01:16.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.771+0000 7f448de00640 1 -- start start 2024-01-16T15:01:16.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.772+0000 7f448de00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4468003ee0 0x7f448815b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:16.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.772+0000 7f448de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44881596d0 0x7f4488157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:16.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.772+0000 7f448de00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44880aff50 0x7f44880b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:16.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.772+0000 7f448de00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4488158480 con 0x7f44881596d0 2024-01-16T15:01:16.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.772+0000 7f448de00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4488159b50 con 0x7f44880aff50 2024-01-16T15:01:16.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.772+0000 7f448de00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44880b0910 con 0x7f4468003ee0 2024-01-16T15:01:16.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.772+0000 7f447de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44881596d0 0x7f4488157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:16.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.773+0000 7f447de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44881596d0 0x7f4488157f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59034/0 (socket says 172.21.15.143:59034) 2024-01-16T15:01:16.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.773+0000 7f447de00640 1 -- 172.21.15.143:0/3476339967 learned_addr learned my addr 172.21.15.143:0/3476339967 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:16.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.773+0000 7f4487400640 1 --2- 172.21.15.143:0/3476339967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44880aff50 0x7f44880b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:16.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.773+0000 7f4486a00640 1 --2- 172.21.15.143:0/3476339967 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4468003ee0 0x7f448815b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:16.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.773+0000 7f447de00640 1 -- 172.21.15.143:0/3476339967 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4468003ee0 msgr2=0x7f448815b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:16.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.773+0000 7f447de00640 1 --2- 172.21.15.143:0/3476339967 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4468003ee0 0x7f448815b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:16.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.773+0000 7f447de00640 1 -- 172.21.15.143:0/3476339967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44880aff50 msgr2=0x7f44880b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:16.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.773+0000 7f447de00640 1 --2- 172.21.15.143:0/3476339967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44880aff50 0x7f44880b03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:16.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.773+0000 7f447de00640 1 -- 172.21.15.143:0/3476339967 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4488152e40 con 0x7f44881596d0 2024-01-16T15:01:16.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.773+0000 7f4486a00640 1 --2- 172.21.15.143:0/3476339967 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4468003ee0 0x7f448815b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:01:16.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.773+0000 7f447de00640 1 --2- 172.21.15.143:0/3476339967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44881596d0 0x7f4488157f40 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4470035040 tx=0x7f44700012b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:16.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.774+0000 7f447fe00640 1 -- 172.21.15.143:0/3476339967 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470006ea0 con 0x7f44881596d0 2024-01-16T15:01:16.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.774+0000 7f447fe00640 1 -- 172.21.15.143:0/3476339967 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4470005070 con 0x7f44881596d0 2024-01-16T15:01:16.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.774+0000 7f447fe00640 1 -- 172.21.15.143:0/3476339967 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44700053e0 con 0x7f44881596d0 2024-01-16T15:01:16.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.774+0000 7f448de00640 1 -- 172.21.15.143:0/3476339967 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44881f1010 con 0x7f44881596d0 2024-01-16T15:01:16.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.774+0000 7f448de00640 1 -- 172.21.15.143:0/3476339967 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f448815a600 con 0x7f44881596d0 2024-01-16T15:01:16.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.775+0000 7f448de00640 1 -- 172.21.15.143:0/3476339967 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44881f11a0 con 0x7f44881596d0 2024-01-16T15:01:16.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.776+0000 7f447fe00640 1 -- 172.21.15.143:0/3476339967 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f4470039050 con 0x7f44881596d0 2024-01-16T15:01:16.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.777+0000 7f447fe00640 1 --2- 172.21.15.143:0/3476339967 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f446006b2f0 0x7f446006d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:16.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.777+0000 7f447fe00640 1 -- 172.21.15.143:0/3476339967 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f44700aac30 con 0x7f44881596d0 2024-01-16T15:01:16.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.777+0000 7f4486a00640 1 --2- 172.21.15.143:0/3476339967 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f446006b2f0 0x7f446006d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:16.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.778+0000 7f4486a00640 1 --2- 172.21.15.143:0/3476339967 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f446006b2f0 0x7f446006d7b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4478001eb0 tx=0x7f4478009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:16.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.781+0000 7f447fe00640 1 -- 172.21.15.143:0/3476339967 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4470075df0 con 0x7f44881596d0 2024-01-16T15:01:16.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:16.976+0000 7f448de00640 1 -- 172.21.15.143:0/3476339967 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f4488152e20 con 0x7f44881596d0 2024-01-16T15:01:17.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:17.829+0000 7f447fe00640 1 -- 172.21.15.143:0/3476339967 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v31) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4470079410 con 0x7f44881596d0 2024-01-16T15:01:17.811 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' created 2024-01-16T15:01:17.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:17.834+0000 7f448de00640 1 -- 172.21.15.143:0/3476339967 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f446006b2f0 msgr2=0x7f446006d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:17.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:17.834+0000 7f448de00640 1 --2- 172.21.15.143:0/3476339967 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f446006b2f0 0x7f446006d7b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4478001eb0 tx=0x7f4478009040 comp rx=0 tx=0).stop 2024-01-16T15:01:17.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:17.834+0000 7f448de00640 1 -- 172.21.15.143:0/3476339967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44881596d0 msgr2=0x7f4488157f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:17.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:17.834+0000 7f448de00640 1 --2- 172.21.15.143:0/3476339967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44881596d0 0x7f4488157f40 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4470035040 tx=0x7f44700012b0 comp rx=0 tx=0).stop 2024-01-16T15:01:17.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:17.834+0000 7f448de00640 1 -- 172.21.15.143:0/3476339967 shutdown_connections 2024-01-16T15:01:17.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:17.834+0000 7f448de00640 1 --2- 172.21.15.143:0/3476339967 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f446006b2f0 0x7f446006d7b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:17.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:17.834+0000 7f448de00640 1 --2- 172.21.15.143:0/3476339967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44880aff50 0x7f44880b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:17.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:17.834+0000 7f448de00640 1 --2- 172.21.15.143:0/3476339967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44881596d0 0x7f4488157f40 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:17.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:17.834+0000 7f448de00640 1 --2- 172.21.15.143:0/3476339967 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4468003ee0 0x7f448815b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:17.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:17.834+0000 7f448de00640 1 -- 172.21.15.143:0/3476339967 >> 172.21.15.143:0/3476339967 conn(0x7f448806a920 msgr2=0x7f44880af560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:17.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:17.834+0000 7f448de00640 1 -- 172.21.15.143:0/3476339967 shutdown_connections 2024-01-16T15:01:17.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:17.834+0000 7f448de00640 1 -- 172.21.15.143:0/3476339967 wait complete. 2024-01-16T15:01:17.842 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-16T15:01:18.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.067+0000 7f27c9e00640 1 Processor -- start 2024-01-16T15:01:18.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.067+0000 7f27c9e00640 1 -- start start 2024-01-16T15:01:18.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.069+0000 7f27c9e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27c415b440 0x7f27c40cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:18.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.069+0000 7f27c9e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f27c40c59b0 0x7f27c406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:18.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.069+0000 7f27c9e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27c40cc3a0 con 0x7f27c40cbf20 2024-01-16T15:01:18.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.069+0000 7f27c9e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27c415b820 con 0x7f27c40c59b0 2024-01-16T15:01:18.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.069+0000 7f27c9e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27c415eea0 con 0x7f27c415b440 2024-01-16T15:01:18.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.069+0000 7f27b9e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f27c40cbf20 0x7f27c40c52a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57762/0 (socket says 172.21.15.143:57762) 2024-01-16T15:01:18.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.069+0000 7f27b9e00640 1 -- 172.21.15.143:0/2835003328 learned_addr learned my addr 172.21.15.143:0/2835003328 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:01:18.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.069+0000 7f27c2a00640 1 --2- 172.21.15.143:0/2835003328 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27c415b440 0x7f27c40cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:18.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.070+0000 7f27c3400640 1 --2- 172.21.15.143:0/2835003328 >> v2:172.21.15.204:3300/0 conn(0x7f27c40c59b0 0x7f27c406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:18.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.070+0000 7f27c2000640 1 -- 172.21.15.143:0/2835003328 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 345731895 0 0) 0x7f27ac001110 con 0x7f27c40cbf20 2024-01-16T15:01:18.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.070+0000 7f27c2000640 1 -- 172.21.15.143:0/2835003328 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27a4003520 con 0x7f27c40cbf20 2024-01-16T15:01:18.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.070+0000 7f27c2000640 1 -- 172.21.15.143:0/2835003328 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3530391001 0 0) 0x7f27a4003520 con 0x7f27c40cbf20 2024-01-16T15:01:18.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.070+0000 7f27c2000640 1 -- 172.21.15.143:0/2835003328 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27c415b440 msgr2=0x7f27c40cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:18.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.070+0000 7f27c2000640 1 --2- 172.21.15.143:0/2835003328 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27c415b440 0x7f27c40cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:18.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.070+0000 7f27c2000640 1 -- 172.21.15.143:0/2835003328 >> v2:172.21.15.204:3300/0 conn(0x7f27c40c59b0 msgr2=0x7f27c406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:18.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.070+0000 7f27c2000640 1 --2- 172.21.15.143:0/2835003328 >> v2:172.21.15.204:3300/0 conn(0x7f27c40c59b0 0x7f27c406c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:18.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.070+0000 7f27c2000640 1 -- 172.21.15.143:0/2835003328 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c40abf30 con 0x7f27c40cbf20 2024-01-16T15:01:18.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.070+0000 7f27c2000640 1 -- 172.21.15.143:0/2835003328 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7f27ac001800 con 0x7f27c40cbf20 2024-01-16T15:01:18.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.070+0000 7f27c2000640 1 -- 172.21.15.143:0/2835003328 >> v1:172.21.15.143:6789/0 conn(0x7f27c40cbf20 legacy=0x7f27c40c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:18.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.070+0000 7f27c2000640 1 --2- 172.21.15.143:0/2835003328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27a4003970 0x7f27a4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:18.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.070+0000 7f27c2000640 1 --2- 172.21.15.143:0/2835003328 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27a40047c0 0x7f27a4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:18.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.070+0000 7f27c2000640 1 --2- 172.21.15.143:0/2835003328 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27c415b440 0x7f27a4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:18.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.070+0000 7f27c2000640 1 -- 172.21.15.143:0/2835003328 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c415b820 con 0x7f27a40047c0 2024-01-16T15:01:18.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.070+0000 7f27c2000640 1 -- 172.21.15.143:0/2835003328 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c415eea0 con 0x7f27a4003970 2024-01-16T15:01:18.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.070+0000 7f27c2000640 1 -- 172.21.15.143:0/2835003328 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27a400b980 con 0x7f27c415b440 2024-01-16T15:01:18.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.070+0000 7f27b9e00640 1 --2- 172.21.15.143:0/2835003328 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27a40047c0 0x7f27a4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:18.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.071+0000 7f27c2a00640 1 --2- 172.21.15.143:0/2835003328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27a4003970 0x7f27a4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:18.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.071+0000 7f27c3400640 1 --2- 172.21.15.143:0/2835003328 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27c415b440 0x7f27a4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:18.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.071+0000 7f27b9e00640 1 -- 172.21.15.143:0/2835003328 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27c415b440 msgr2=0x7f27a4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:18.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.071+0000 7f27b9e00640 1 --2- 172.21.15.143:0/2835003328 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27c415b440 0x7f27a4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:18.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.071+0000 7f27b9e00640 1 -- 172.21.15.143:0/2835003328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27a4003970 msgr2=0x7f27a4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:18.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.071+0000 7f27b9e00640 1 --2- 172.21.15.143:0/2835003328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27a4003970 0x7f27a4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:18.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.071+0000 7f27b9e00640 1 -- 172.21.15.143:0/2835003328 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a4003520 con 0x7f27a40047c0 2024-01-16T15:01:18.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.072+0000 7f27b9e00640 1 --2- 172.21.15.143:0/2835003328 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27a40047c0 0x7f27a4004c40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f27ac00c240 tx=0x7f27a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e47bf5f27ab3ec1a server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:18.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.072+0000 7f27c2000640 1 -- 172.21.15.143:0/2835003328 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ac005ad0 con 0x7f27a40047c0 2024-01-16T15:01:18.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.072+0000 7f27c2000640 1 -- 172.21.15.143:0/2835003328 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27ac005c50 con 0x7f27a40047c0 2024-01-16T15:01:18.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.072+0000 7f27c2000640 1 -- 172.21.15.143:0/2835003328 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ac02c8d0 con 0x7f27a40047c0 2024-01-16T15:01:18.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.073+0000 7f27c9e00640 1 -- 172.21.15.143:0/2835003328 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27a40047c0 msgr2=0x7f27a4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:18.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.073+0000 7f27c9e00640 1 --2- 172.21.15.143:0/2835003328 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27a40047c0 0x7f27a4004c40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f27ac00c240 tx=0x7f27a40057f0 comp rx=0 tx=0).stop 2024-01-16T15:01:18.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.073+0000 7f27c3400640 1 -- 172.21.15.143:0/2835003328 reap_dead start 2024-01-16T15:01:18.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.073+0000 7f27c9e00640 1 -- 172.21.15.143:0/2835003328 shutdown_connections 2024-01-16T15:01:18.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.073+0000 7f27c9e00640 1 -- 172.21.15.143:0/2835003328 >> 172.21.15.143:0/2835003328 conn(0x7f27c40b0c50 msgr2=0x7f27c40b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:18.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.073+0000 7f27c9e00640 1 -- 172.21.15.143:0/2835003328 shutdown_connections 2024-01-16T15:01:18.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.073+0000 7f27c9e00640 1 -- 172.21.15.143:0/2835003328 wait complete. 2024-01-16T15:01:18.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.075+0000 7f27c9e00640 1 Processor -- start 2024-01-16T15:01:18.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.075+0000 7f27c9e00640 1 -- start start 2024-01-16T15:01:18.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.076+0000 7f27c9e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27a40047c0 0x7f27c41ed190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:18.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.076+0000 7f27c9e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27c40999d0 0x7f27c41f3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:18.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.076+0000 7f27c9e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27c409bb10 0x7f27c409bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:18.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.076+0000 7f27c9e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c40c8c60 con 0x7f27a40047c0 2024-01-16T15:01:18.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.076+0000 7f27c9e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c409c4d0 con 0x7f27c409bb10 2024-01-16T15:01:18.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.076+0000 7f27c9e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27c409afa0 con 0x7f27c40999d0 2024-01-16T15:01:18.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.076+0000 7f27c2a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27a40047c0 0x7f27c41ed190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:18.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.076+0000 7f27c2a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27a40047c0 0x7f27c41ed190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59062/0 (socket says 172.21.15.143:59062) 2024-01-16T15:01:18.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.076+0000 7f27c2a00640 1 -- 172.21.15.143:0/1167800602 learned_addr learned my addr 172.21.15.143:0/1167800602 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:18.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.077+0000 7f27b9e00640 1 --2- 172.21.15.143:0/1167800602 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27c40999d0 0x7f27c41f3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:18.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.077+0000 7f27c3400640 1 --2- 172.21.15.143:0/1167800602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27c409bb10 0x7f27c409bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:18.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.077+0000 7f27c2a00640 1 -- 172.21.15.143:0/1167800602 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27c40999d0 msgr2=0x7f27c41f3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:18.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.077+0000 7f27c2a00640 1 --2- 172.21.15.143:0/1167800602 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27c40999d0 0x7f27c41f3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:18.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.077+0000 7f27c2a00640 1 -- 172.21.15.143:0/1167800602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27c409bb10 msgr2=0x7f27c409bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:18.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.077+0000 7f27c2a00640 1 --2- 172.21.15.143:0/1167800602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27c409bb10 0x7f27c409bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:18.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.077+0000 7f27c2a00640 1 -- 172.21.15.143:0/1167800602 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c409ab10 con 0x7f27a40047c0 2024-01-16T15:01:18.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.077+0000 7f27c3400640 1 --2- 172.21.15.143:0/1167800602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27c409bb10 0x7f27c409bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:01:18.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.077+0000 7f27c2a00640 1 --2- 172.21.15.143:0/1167800602 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27a40047c0 0x7f27c41ed190 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f27b4011570 tx=0x7f27b4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:18.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.077+0000 7f27bbe00640 1 -- 172.21.15.143:0/1167800602 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b4015930 con 0x7f27a40047c0 2024-01-16T15:01:18.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.077+0000 7f27bbe00640 1 -- 172.21.15.143:0/1167800602 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b4015a90 con 0x7f27a40047c0 2024-01-16T15:01:18.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.077+0000 7f27bbe00640 1 -- 172.21.15.143:0/1167800602 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b4015e60 con 0x7f27a40047c0 2024-01-16T15:01:18.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.077+0000 7f27c9e00640 1 -- 172.21.15.143:0/1167800602 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27c41f4090 con 0x7f27a40047c0 2024-01-16T15:01:18.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.078+0000 7f27c9e00640 1 -- 172.21.15.143:0/1167800602 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c409a1e0 con 0x7f27a40047c0 2024-01-16T15:01:18.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.079+0000 7f27c9e00640 1 -- 172.21.15.143:0/1167800602 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27c404e8b0 con 0x7f27a40047c0 2024-01-16T15:01:18.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.081+0000 7f27bbe00640 1 -- 172.21.15.143:0/1167800602 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f27b400f460 con 0x7f27a40047c0 2024-01-16T15:01:18.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.081+0000 7f27bbe00640 1 --2- 172.21.15.143:0/1167800602 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f279c06b2c0 0x7f279c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:18.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.082+0000 7f27bbe00640 1 -- 172.21.15.143:0/1167800602 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f27b408a210 con 0x7f27a40047c0 2024-01-16T15:01:18.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.082+0000 7f27b9e00640 1 --2- 172.21.15.143:0/1167800602 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f279c06b2c0 0x7f279c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:18.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.083+0000 7f27b9e00640 1 --2- 172.21.15.143:0/1167800602 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f279c06b2c0 0x7f279c06d780 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f27ac034000 tx=0x7f27ac007000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:18.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.086+0000 7f27bbe00640 1 -- 172.21.15.143:0/1167800602 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f27b4055340 con 0x7f27a40047c0 2024-01-16T15:01:18.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.273+0000 7f27c9e00640 1 -- 172.21.15.143:0/1167800602 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f27c406dc80 con 0x7f27a40047c0 2024-01-16T15:01:18.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.835+0000 7f27bbe00640 1 -- 172.21.15.143:0/1167800602 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v32) v1 ==== 170+0+0 (secure 0 0 0) 0x7f27b4058960 con 0x7f27a40047c0 2024-01-16T15:01:18.835 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' created 2024-01-16T15:01:18.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.840+0000 7f27c9e00640 1 -- 172.21.15.143:0/1167800602 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f279c06b2c0 msgr2=0x7f279c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:18.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.840+0000 7f27c9e00640 1 --2- 172.21.15.143:0/1167800602 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f279c06b2c0 0x7f279c06d780 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f27ac034000 tx=0x7f27ac007000 comp rx=0 tx=0).stop 2024-01-16T15:01:18.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.840+0000 7f27c9e00640 1 -- 172.21.15.143:0/1167800602 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27a40047c0 msgr2=0x7f27c41ed190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:18.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.840+0000 7f27c9e00640 1 --2- 172.21.15.143:0/1167800602 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27a40047c0 0x7f27c41ed190 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f27b4011570 tx=0x7f27b4011a30 comp rx=0 tx=0).stop 2024-01-16T15:01:18.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.841+0000 7f27c9e00640 1 -- 172.21.15.143:0/1167800602 shutdown_connections 2024-01-16T15:01:18.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.841+0000 7f27c9e00640 1 --2- 172.21.15.143:0/1167800602 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f279c06b2c0 0x7f279c06d780 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:18.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.841+0000 7f27c9e00640 1 --2- 172.21.15.143:0/1167800602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27c409bb10 0x7f27c409bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:18.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.841+0000 7f27c9e00640 1 --2- 172.21.15.143:0/1167800602 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27c40999d0 0x7f27c41f3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:18.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.841+0000 7f27c9e00640 1 --2- 172.21.15.143:0/1167800602 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27a40047c0 0x7f27c41ed190 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:18.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.841+0000 7f27c9e00640 1 -- 172.21.15.143:0/1167800602 >> 172.21.15.143:0/1167800602 conn(0x7f27c40b0c50 msgr2=0x7f27c40ad540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:18.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.843+0000 7f27c9e00640 1 -- 172.21.15.143:0/1167800602 shutdown_connections 2024-01-16T15:01:18.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:18.843+0000 7f27c9e00640 1 -- 172.21.15.143:0/1167800602 wait complete. 2024-01-16T15:01:18.854 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-16T15:01:19.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.052+0000 7f8982a00640 1 Processor -- start 2024-01-16T15:01:19.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.052+0000 7f8982a00640 1 -- start start 2024-01-16T15:01:19.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.054+0000 7f8982a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f897c15b440 0x7f897c0cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:19.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.054+0000 7f8982a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f897c0cbf20 0x7f897c0c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:19.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.054+0000 7f8982a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f897c15eea0 con 0x7f897c0c57e0 2024-01-16T15:01:19.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.054+0000 7f8982a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f897c0cc3d0 con 0x7f897c0cbf20 2024-01-16T15:01:19.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.054+0000 7f8982a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f897c15b820 con 0x7f897c15b440 2024-01-16T15:01:19.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.054+0000 7f8973200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f897c0cbf20 0x7f897c0c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:19.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.055+0000 7f8973200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f897c0cbf20 0x7f897c0c52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59066/0 (socket says 172.21.15.143:59066) 2024-01-16T15:01:19.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.055+0000 7f8973200640 1 -- 172.21.15.143:0/2146844023 learned_addr learned my addr 172.21.15.143:0/2146844023 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:19.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.055+0000 7f897be00640 1 --2- 172.21.15.143:0/2146844023 >> v2:172.21.15.204:3300/0 conn(0x7f897c15b440 0x7f897c0cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:19.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.055+0000 7f8973200640 1 -- 172.21.15.143:0/2146844023 >> v2:172.21.15.204:3300/0 conn(0x7f897c15b440 msgr2=0x7f897c0cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:19.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.055+0000 7f8973200640 1 --2- 172.21.15.143:0/2146844023 >> v2:172.21.15.204:3300/0 conn(0x7f897c15b440 0x7f897c0cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:19.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.055+0000 7f8973200640 1 -- 172.21.15.143:0/2146844023 >> v1:172.21.15.204:6789/0 conn(0x7f897c0c57e0 legacy=0x7f897c0c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:19.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.055+0000 7f8973200640 1 -- 172.21.15.143:0/2146844023 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f897c06d600 con 0x7f897c0cbf20 2024-01-16T15:01:19.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.056+0000 7f8973200640 1 --2- 172.21.15.143:0/2146844023 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f897c0cbf20 0x7f897c0c52a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f897c06e770 tx=0x7f896402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec477f24e2be7432 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:19.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.056+0000 7f897b400640 1 -- 172.21.15.143:0/2146844023 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89640317e0 con 0x7f897c0cbf20 2024-01-16T15:01:19.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.056+0000 7f897b400640 1 -- 172.21.15.143:0/2146844023 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8964031940 con 0x7f897c0cbf20 2024-01-16T15:01:19.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.056+0000 7f897b400640 1 -- 172.21.15.143:0/2146844023 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8964031cd0 con 0x7f897c0cbf20 2024-01-16T15:01:19.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.057+0000 7f8982a00640 1 -- 172.21.15.143:0/2146844023 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f897c0cbf20 msgr2=0x7f897c0c52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:19.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.057+0000 7f8982a00640 1 --2- 172.21.15.143:0/2146844023 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f897c0cbf20 0x7f897c0c52a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f897c06e770 tx=0x7f896402b880 comp rx=0 tx=0).stop 2024-01-16T15:01:19.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.057+0000 7f8982a00640 1 -- 172.21.15.143:0/2146844023 shutdown_connections 2024-01-16T15:01:19.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.057+0000 7f8982a00640 1 --2- 172.21.15.143:0/2146844023 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f897c0cbf20 0x7f897c0c52a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:19.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.057+0000 7f8982a00640 1 --2- 172.21.15.143:0/2146844023 >> v2:172.21.15.204:3300/0 conn(0x7f897c15b440 0x7f897c0cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:19.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.057+0000 7f8982a00640 1 -- 172.21.15.143:0/2146844023 >> 172.21.15.143:0/2146844023 conn(0x7f897c0b0c50 msgr2=0x7f897c0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:19.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.057+0000 7f8982a00640 1 -- 172.21.15.143:0/2146844023 shutdown_connections 2024-01-16T15:01:19.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.057+0000 7f8982a00640 1 -- 172.21.15.143:0/2146844023 wait complete. 2024-01-16T15:01:19.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.059+0000 7f8982a00640 1 Processor -- start 2024-01-16T15:01:19.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.059+0000 7f8982a00640 1 -- start start 2024-01-16T15:01:19.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.060+0000 7f8982a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f897c0c57e0 0x7f897c154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:19.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.060+0000 7f8982a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f897c0cbf20 0x7f897c0b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:19.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.060+0000 7f8982a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f897c15b440 0x7f897c0ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:19.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.060+0000 7f8982a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897c155030 con 0x7f897c15b440 2024-01-16T15:01:19.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.060+0000 7f8982a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897c0ce800 con 0x7f897c0cbf20 2024-01-16T15:01:19.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.060+0000 7f8982a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f897c0ae970 con 0x7f897c0c57e0 2024-01-16T15:01:19.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.060+0000 7f8980a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f897c15b440 0x7f897c0ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:19.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.060+0000 7f8980a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f897c15b440 0x7f897c0ad1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59080/0 (socket says 172.21.15.143:59080) 2024-01-16T15:01:19.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.060+0000 7f8980a00640 1 -- 172.21.15.143:0/836955510 learned_addr learned my addr 172.21.15.143:0/836955510 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:19.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.061+0000 7f8973200640 1 --2- 172.21.15.143:0/836955510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f897c0cbf20 0x7f897c0b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:19.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.061+0000 7f897be00640 1 --2- 172.21.15.143:0/836955510 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f897c0c57e0 0x7f897c154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:19.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.061+0000 7f8980a00640 1 -- 172.21.15.143:0/836955510 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f897c0c57e0 msgr2=0x7f897c154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:19.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.061+0000 7f8980a00640 1 --2- 172.21.15.143:0/836955510 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f897c0c57e0 0x7f897c154af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:19.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.061+0000 7f8980a00640 1 -- 172.21.15.143:0/836955510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f897c0cbf20 msgr2=0x7f897c0b1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:19.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.061+0000 7f8980a00640 1 --2- 172.21.15.143:0/836955510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f897c0cbf20 0x7f897c0b1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:19.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.061+0000 7f8980a00640 1 -- 172.21.15.143:0/836955510 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89640094d0 con 0x7f897c15b440 2024-01-16T15:01:19.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.061+0000 7f8980a00640 1 --2- 172.21.15.143:0/836955510 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f897c15b440 0x7f897c0ad1a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f897400c440 tx=0x7f897400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:19.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.061+0000 7f8978c00640 1 -- 172.21.15.143:0/836955510 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8974011790 con 0x7f897c15b440 2024-01-16T15:01:19.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.061+0000 7f8978c00640 1 -- 172.21.15.143:0/836955510 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89740118f0 con 0x7f897c15b440 2024-01-16T15:01:19.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.061+0000 7f8978c00640 1 -- 172.21.15.143:0/836955510 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8974011ca0 con 0x7f897c15b440 2024-01-16T15:01:19.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.061+0000 7f8982a00640 1 -- 172.21.15.143:0/836955510 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f897c0b2250 con 0x7f897c15b440 2024-01-16T15:01:19.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.062+0000 7f8982a00640 1 -- 172.21.15.143:0/836955510 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f897c09c7e0 con 0x7f897c15b440 2024-01-16T15:01:19.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.063+0000 7f8982a00640 1 -- 172.21.15.143:0/836955510 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f897c04e8b0 con 0x7f897c15b440 2024-01-16T15:01:19.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.064+0000 7f8978c00640 1 -- 172.21.15.143:0/836955510 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f897401c460 con 0x7f897c15b440 2024-01-16T15:01:19.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.065+0000 7f8978c00640 1 --2- 172.21.15.143:0/836955510 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f895406b2c0 0x7f895406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:19.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.065+0000 7f8978c00640 1 -- 172.21.15.143:0/836955510 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f897408a3c0 con 0x7f897c15b440 2024-01-16T15:01:19.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.065+0000 7f897be00640 1 --2- 172.21.15.143:0/836955510 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f895406b2c0 0x7f895406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:19.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.066+0000 7f897be00640 1 --2- 172.21.15.143:0/836955510 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f895406b2c0 0x7f895406d780 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f896c0099c0 tx=0x7f896c003990 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:19.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.069+0000 7f8978c00640 1 -- 172.21.15.143:0/836955510 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8974055400 con 0x7f897c15b440 2024-01-16T15:01:19.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.288+0000 7f8982a00640 1 -- 172.21.15.143:0/836955510 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f897c06dc80 con 0x7f897c15b440 2024-01-16T15:01:19.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.882+0000 7f8978c00640 1 -- 172.21.15.143:0/836955510 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7f8974058a20 con 0x7f897c15b440 2024-01-16T15:01:19.862 INFO:teuthology.orchestra.run.smithi143.stderr:new fs with metadata pool 4 and data pool 5 2024-01-16T15:01:19.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.894+0000 7f8982a00640 1 -- 172.21.15.143:0/836955510 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f895406b2c0 msgr2=0x7f895406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:19.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.894+0000 7f8982a00640 1 --2- 172.21.15.143:0/836955510 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f895406b2c0 0x7f895406d780 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f896c0099c0 tx=0x7f896c003990 comp rx=0 tx=0).stop 2024-01-16T15:01:19.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.894+0000 7f8982a00640 1 -- 172.21.15.143:0/836955510 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f897c15b440 msgr2=0x7f897c0ad1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:19.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.894+0000 7f8982a00640 1 --2- 172.21.15.143:0/836955510 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f897c15b440 0x7f897c0ad1a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f897400c440 tx=0x7f897400c900 comp rx=0 tx=0).stop 2024-01-16T15:01:19.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.896+0000 7f8982a00640 1 -- 172.21.15.143:0/836955510 shutdown_connections 2024-01-16T15:01:19.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.896+0000 7f8982a00640 1 --2- 172.21.15.143:0/836955510 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f895406b2c0 0x7f895406d780 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:19.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.896+0000 7f8982a00640 1 --2- 172.21.15.143:0/836955510 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f897c15b440 0x7f897c0ad1a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:19.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.896+0000 7f8982a00640 1 --2- 172.21.15.143:0/836955510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f897c0cbf20 0x7f897c0b1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:19.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.896+0000 7f8982a00640 1 --2- 172.21.15.143:0/836955510 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f897c0c57e0 0x7f897c154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:19.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.896+0000 7f8982a00640 1 -- 172.21.15.143:0/836955510 >> 172.21.15.143:0/836955510 conn(0x7f897c0b0c50 msgr2=0x7f897c070d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:19.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.898+0000 7f8982a00640 1 -- 172.21.15.143:0/836955510 shutdown_connections 2024-01-16T15:01:19.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:19.898+0000 7f8982a00640 1 -- 172.21.15.143:0/836955510 wait complete. 2024-01-16T15:01:19.909 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T15:01:20.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.104+0000 7f0cdbc00640 1 Processor -- start 2024-01-16T15:01:20.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.104+0000 7f0cdbc00640 1 -- start start 2024-01-16T15:01:20.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.106+0000 7f0cdbc00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0cd40cf450 0x7f0cd40cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:20.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.106+0000 7f0cdbc00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0cd40cdfb0 0x7f0cd40ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:20.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.106+0000 7f0cdbc00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cd40cece0 con 0x7f0cd415c7f0 2024-01-16T15:01:20.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.106+0000 7f0cdbc00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cd40cfe40 con 0x7f0cd40cf450 2024-01-16T15:01:20.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.106+0000 7f0cdbc00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cd4156010 con 0x7f0cd40cdfb0 2024-01-16T15:01:20.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.106+0000 7f0cd9200640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f0cd415c7f0 0x7f0cd406db20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57778/0 (socket says 172.21.15.143:57778) 2024-01-16T15:01:20.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.106+0000 7f0cd9200640 1 -- 172.21.15.143:0/4266220593 learned_addr learned my addr 172.21.15.143:0/4266220593 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:01:20.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.106+0000 7f0cd3e00640 1 --2- 172.21.15.143:0/4266220593 >> v2:172.21.15.143:3300/0 conn(0x7f0cd40cf450 0x7f0cd40cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:20.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.106+0000 7f0cd3400640 1 -- 172.21.15.143:0/4266220593 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1497690372 0 0) 0x7f0cd40cece0 con 0x7f0cd415c7f0 2024-01-16T15:01:20.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.107+0000 7f0cd9c00640 1 --2- 172.21.15.143:0/4266220593 >> v2:172.21.15.204:3300/0 conn(0x7f0cd40cdfb0 0x7f0cd40ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:20.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.107+0000 7f0cd3400640 1 -- 172.21.15.143:0/4266220593 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cb4003520 con 0x7f0cd415c7f0 2024-01-16T15:01:20.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.107+0000 7f0cd3e00640 1 -- 172.21.15.143:0/4266220593 >> v2:172.21.15.204:3300/0 conn(0x7f0cd40cdfb0 msgr2=0x7f0cd40ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:20.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.107+0000 7f0cd3e00640 1 --2- 172.21.15.143:0/4266220593 >> v2:172.21.15.204:3300/0 conn(0x7f0cd40cdfb0 0x7f0cd40ce3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:20.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.107+0000 7f0cd3e00640 1 -- 172.21.15.143:0/4266220593 >> v1:172.21.15.143:6789/0 conn(0x7f0cd415c7f0 legacy=0x7f0cd406db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:20.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.107+0000 7f0cd3e00640 1 -- 172.21.15.143:0/4266220593 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd406d040 con 0x7f0cd40cf450 2024-01-16T15:01:20.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.107+0000 7f0cd9c00640 1 --2- 172.21.15.143:0/4266220593 >> v2:172.21.15.204:3300/0 conn(0x7f0cd40cdfb0 0x7f0cd40ce3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:01:20.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.108+0000 7f0cd3e00640 1 --2- 172.21.15.143:0/4266220593 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cd40cf450 0x7f0cd40cf900 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0cd40d00e0 tx=0x7f0cc402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ca014a8b0fd3d0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:20.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.108+0000 7f0cd3400640 1 -- 172.21.15.143:0/4266220593 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc402cc70 con 0x7f0cd40cf450 2024-01-16T15:01:20.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.108+0000 7f0cd3400640 1 -- 172.21.15.143:0/4266220593 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cc402cdd0 con 0x7f0cd40cf450 2024-01-16T15:01:20.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.108+0000 7f0cd3400640 1 -- 172.21.15.143:0/4266220593 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc4034640 con 0x7f0cd40cf450 2024-01-16T15:01:20.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.109+0000 7f0cdbc00640 1 -- 172.21.15.143:0/4266220593 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cd40cf450 msgr2=0x7f0cd40cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:20.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.109+0000 7f0cdbc00640 1 --2- 172.21.15.143:0/4266220593 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cd40cf450 0x7f0cd40cf900 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0cd40d00e0 tx=0x7f0cc402bec0 comp rx=0 tx=0).stop 2024-01-16T15:01:20.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.109+0000 7f0cdbc00640 1 -- 172.21.15.143:0/4266220593 shutdown_connections 2024-01-16T15:01:20.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.109+0000 7f0cdbc00640 1 --2- 172.21.15.143:0/4266220593 >> v2:172.21.15.204:3300/0 conn(0x7f0cd40cdfb0 0x7f0cd40ce3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:20.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.109+0000 7f0cdbc00640 1 --2- 172.21.15.143:0/4266220593 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cd40cf450 0x7f0cd40cf900 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:20.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.109+0000 7f0cdbc00640 1 -- 172.21.15.143:0/4266220593 >> 172.21.15.143:0/4266220593 conn(0x7f0cd406a920 msgr2=0x7f0cd406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:20.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.109+0000 7f0cdbc00640 1 -- 172.21.15.143:0/4266220593 shutdown_connections 2024-01-16T15:01:20.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.109+0000 7f0cdbc00640 1 -- 172.21.15.143:0/4266220593 wait complete. 2024-01-16T15:01:20.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.111+0000 7f0cdbc00640 1 Processor -- start 2024-01-16T15:01:20.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.111+0000 7f0cdbc00640 1 -- start start 2024-01-16T15:01:20.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.112+0000 7f0cdbc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cd40cdfb0 0x7f0cd4144e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:20.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.112+0000 7f0cdbc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0cd40cf450 0x7f0cd41f2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:20.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.112+0000 7f0cdbc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cd415c7f0 0x7f0cd40c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:20.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.112+0000 7f0cdbc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd40b2640 con 0x7f0cd40cdfb0 2024-01-16T15:01:20.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.112+0000 7f0cdbc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd415aea0 con 0x7f0cd415c7f0 2024-01-16T15:01:20.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.112+0000 7f0cdbc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd40b2890 con 0x7f0cd40cf450 2024-01-16T15:01:20.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.112+0000 7f0cd9200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cd40cdfb0 0x7f0cd4144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:20.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.112+0000 7f0cd9200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cd40cdfb0 0x7f0cd4144e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59108/0 (socket says 172.21.15.143:59108) 2024-01-16T15:01:20.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.112+0000 7f0cd9200640 1 -- 172.21.15.143:0/3289961697 learned_addr learned my addr 172.21.15.143:0/3289961697 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:20.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.112+0000 7f0cd9c00640 1 --2- 172.21.15.143:0/3289961697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cd415c7f0 0x7f0cd40c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:20.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.112+0000 7f0cd9200640 1 -- 172.21.15.143:0/3289961697 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0cd40cf450 msgr2=0x7f0cd41f2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:20.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.112+0000 7f0cd9200640 1 --2- 172.21.15.143:0/3289961697 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0cd40cf450 0x7f0cd41f2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:20.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.112+0000 7f0cd9200640 1 -- 172.21.15.143:0/3289961697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cd415c7f0 msgr2=0x7f0cd40c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:20.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.112+0000 7f0cd9200640 1 --2- 172.21.15.143:0/3289961697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cd415c7f0 0x7f0cd40c8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:20.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.112+0000 7f0cd9200640 1 -- 172.21.15.143:0/3289961697 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc4009a70 con 0x7f0cd40cdfb0 2024-01-16T15:01:20.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.113+0000 7f0cd9200640 1 --2- 172.21.15.143:0/3289961697 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cd40cdfb0 0x7f0cd4144e70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc00c390 tx=0x7f0cbc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:20.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.113+0000 7f0cd0c00640 1 -- 172.21.15.143:0/3289961697 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc005ce0 con 0x7f0cd40cdfb0 2024-01-16T15:01:20.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.113+0000 7f0cd0c00640 1 -- 172.21.15.143:0/3289961697 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cbc005e40 con 0x7f0cd40cdfb0 2024-01-16T15:01:20.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.113+0000 7f0cdbc00640 1 -- 172.21.15.143:0/3289961697 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd40b2b70 con 0x7f0cd40cdfb0 2024-01-16T15:01:20.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.113+0000 7f0cd0c00640 1 -- 172.21.15.143:0/3289961697 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc0072f0 con 0x7f0cd40cdfb0 2024-01-16T15:01:20.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.113+0000 7f0cdbc00640 1 -- 172.21.15.143:0/3289961697 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd41f2f70 con 0x7f0cd40cdfb0 2024-01-16T15:01:20.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.114+0000 7f0cdbc00640 1 -- 172.21.15.143:0/3289961697 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cd4067470 con 0x7f0cd40cdfb0 2024-01-16T15:01:20.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.115+0000 7f0cd0c00640 1 -- 172.21.15.143:0/3289961697 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f0cbc007450 con 0x7f0cd40cdfb0 2024-01-16T15:01:20.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.116+0000 7f0cd0c00640 1 --2- 172.21.15.143:0/3289961697 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0cac06b2c0 0x7f0cac06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:20.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.116+0000 7f0cd0c00640 1 -- 172.21.15.143:0/3289961697 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0cbc089200 con 0x7f0cd40cdfb0 2024-01-16T15:01:20.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.116+0000 7f0cd3e00640 1 --2- 172.21.15.143:0/3289961697 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0cac06b2c0 0x7f0cac06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:20.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.117+0000 7f0cd3e00640 1 --2- 172.21.15.143:0/3289961697 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0cac06b2c0 0x7f0cac06d780 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4036000 tx=0x7f0cc402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:20.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.121+0000 7f0cd0c00640 1 -- 172.21.15.143:0/3289961697 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0cbc054060 con 0x7f0cd40cdfb0 2024-01-16T15:01:20.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.302+0000 7f0cdbc00640 1 -- 172.21.15.143:0/3289961697 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0cd40cd730 con 0x7f0cd40cdfb0 2024-01-16T15:01:20.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.304+0000 7f0cd0c00640 1 -- 172.21.15.143:0/3289961697 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16642 (secure 0 0 0) 0x7f0cbc057680 con 0x7f0cd40cdfb0 2024-01-16T15:01:20.284 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:01:20.285 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":33,"fsid":"c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0","created":"2024-01-16T14:59:38.532601+0000","modified":"2024-01-16T15:01:19.289691+0000","last_up_change":"2024-01-16T14:59:50.983699+0000","last_in_change":"2024-01-16T14:59:42.056297+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:59:51.727720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-01-16T15:01:17.024625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-01-16T15:01:18.326532+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f4c8891d-ab29-472a-907d-8446f20f7012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6801","nonce":1569293626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6805","nonce":1569293626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6814","nonce":1569293626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6809","nonce":1569293626}]},"public_addr":"172.21.15.143:6801/1569293626","cluster_addr":"172.21.15.143:6805/1569293626","heartbeat_back_addr":"172.21.15.143:6814/1569293626","heartbeat_front_addr":"172.21.15.143:6809/1569293626","state":["exists","up"]},{"osd":1,"uuid":"c4a2788d-8b89-4992-82b2-7f4b9debd509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6804","nonce":1276841168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6808","nonce":1276841168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6813","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6815","nonce":1276841168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6811","nonce":1276841168}]},"public_addr":"172.21.15.143:6804/1276841168","cluster_addr":"172.21.15.143:6808/1276841168","heartbeat_back_addr":"172.21.15.143:6815/1276841168","heartbeat_front_addr":"172.21.15.143:6811/1276841168","state":["exists","up"]},{"osd":2,"uuid":"64645c1d-31b2-4794-b0f2-7ccf8df30393","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6817","nonce":1217670170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6819","nonce":1217670170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6823","nonce":1217670170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6821","nonce":1217670170}]},"public_addr":"172.21.15.143:6817/1217670170","cluster_addr":"172.21.15.143:6819/1217670170","heartbeat_back_addr":"172.21.15.143:6823/1217670170","heartbeat_front_addr":"172.21.15.143:6821/1217670170","state":["exists","up"]},{"osd":3,"uuid":"73674129-a84e-4d31-8247-7d0179ae6df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6825","nonce":3414188012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6827","nonce":3414188012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6831","nonce":3414188012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6829","nonce":3414188012}]},"public_addr":"172.21.15.143:6825/3414188012","cluster_addr":"172.21.15.143:6827/3414188012","heartbeat_back_addr":"172.21.15.143:6831/3414188012","heartbeat_front_addr":"172.21.15.143:6829/3414188012","state":["exists","up"]},{"osd":4,"uuid":"cea28c9b-2f1c-4087-b415-9960789131d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6811","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6812","nonce":1070255716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6816","nonce":1070255716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6823","nonce":1070255716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6820","nonce":1070255716}]},"public_addr":"172.21.15.204:6812/1070255716","cluster_addr":"172.21.15.204:6816/1070255716","heartbeat_back_addr":"172.21.15.204:6823/1070255716","heartbeat_front_addr":"172.21.15.204:6820/1070255716","state":["exists","up"]},{"osd":5,"uuid":"33b63560-5574-4e78-983f-b398d67a2cb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6809","nonce":3605417837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6813","nonce":3605417837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6819","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6821","nonce":3605417837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6817","nonce":3605417837}]},"public_addr":"172.21.15.204:6809/3605417837","cluster_addr":"172.21.15.204:6813/3605417837","heartbeat_back_addr":"172.21.15.204:6821/3605417837","heartbeat_front_addr":"172.21.15.204:6817/3605417837","state":["exists","up"]},{"osd":6,"uuid":"df397427-8940-4eee-9c79-eece669d735d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6801","nonce":1085000534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6803","nonce":1085000534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6808","nonce":1085000534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6805","nonce":1085000534}]},"public_addr":"172.21.15.204:6801/1085000534","cluster_addr":"172.21.15.204:6803/1085000534","heartbeat_back_addr":"172.21.15.204:6808/1085000534","heartbeat_front_addr":"172.21.15.204:6805/1085000534","state":["exists","up"]},{"osd":7,"uuid":"a2e5a15c-8312-405a-ac02-f7e1fc9df2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6825","nonce":1337220609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6827","nonce":1337220609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6831","nonce":1337220609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6829","nonce":1337220609}]},"public_addr":"172.21.15.204:6825/1337220609","cluster_addr":"172.21.15.204:6827/1337220609","heartbeat_back_addr":"172.21.15.204:6831/1337220609","heartbeat_front_addr":"172.21.15.204:6829/1337220609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:49.984507+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.395306+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.632998+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.373627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.342920+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.438366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.331926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.406657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T15:01:20.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.307+0000 7f0cdbc00640 1 -- 172.21.15.143:0/3289961697 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0cac06b2c0 msgr2=0x7f0cac06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:20.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.307+0000 7f0cdbc00640 1 --2- 172.21.15.143:0/3289961697 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0cac06b2c0 0x7f0cac06d780 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4036000 tx=0x7f0cc402e000 comp rx=0 tx=0).stop 2024-01-16T15:01:20.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.307+0000 7f0cdbc00640 1 -- 172.21.15.143:0/3289961697 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cd40cdfb0 msgr2=0x7f0cd4144e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:20.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.307+0000 7f0cdbc00640 1 --2- 172.21.15.143:0/3289961697 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cd40cdfb0 0x7f0cd4144e70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc00c390 tx=0x7f0cbc00c850 comp rx=0 tx=0).stop 2024-01-16T15:01:20.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.308+0000 7f0cdbc00640 1 -- 172.21.15.143:0/3289961697 shutdown_connections 2024-01-16T15:01:20.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.308+0000 7f0cdbc00640 1 --2- 172.21.15.143:0/3289961697 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0cac06b2c0 0x7f0cac06d780 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:20.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.308+0000 7f0cdbc00640 1 --2- 172.21.15.143:0/3289961697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cd415c7f0 0x7f0cd40c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:20.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.308+0000 7f0cdbc00640 1 --2- 172.21.15.143:0/3289961697 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0cd40cf450 0x7f0cd41f2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:20.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.308+0000 7f0cdbc00640 1 --2- 172.21.15.143:0/3289961697 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cd40cdfb0 0x7f0cd4144e70 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:20.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.308+0000 7f0cdbc00640 1 -- 172.21.15.143:0/3289961697 >> 172.21.15.143:0/3289961697 conn(0x7f0cd406a920 msgr2=0x7f0cd40af720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:20.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.308+0000 7f0cdbc00640 1 -- 172.21.15.143:0/3289961697 shutdown_connections 2024-01-16T15:01:20.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.308+0000 7f0cdbc00640 1 -- 172.21.15.143:0/3289961697 wait complete. 2024-01-16T15:01:20.304 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T15:01:20.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.520+0000 7f4008400640 1 Processor -- start 2024-01-16T15:01:20.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.521+0000 7f4008400640 1 -- start start 2024-01-16T15:01:20.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.522+0000 7f4008400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f40000cf450 0x7f40000cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:20.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.522+0000 7f4008400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40000cdfb0 0x7f40000ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:20.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.522+0000 7f4008400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40000cece0 con 0x7f400015c7f0 2024-01-16T15:01:20.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.522+0000 7f4008400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40000cfe40 con 0x7f40000cf450 2024-01-16T15:01:20.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.522+0000 7f4008400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4000156010 con 0x7f40000cdfb0 2024-01-16T15:01:20.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.523+0000 7f4005a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f400015c7f0 0x7f400006db20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57782/0 (socket says 172.21.15.143:57782) 2024-01-16T15:01:20.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.523+0000 7f4005a00640 1 -- 172.21.15.143:0/1081516421 learned_addr learned my addr 172.21.15.143:0/1081516421 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:01:20.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.523+0000 7f4006400640 1 --2- 172.21.15.143:0/1081516421 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40000cdfb0 0x7f40000ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:20.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.523+0000 7f4005000640 1 --2- 172.21.15.143:0/1081516421 >> v2:172.21.15.204:3300/0 conn(0x7f40000cf450 0x7f40000cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:20.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.523+0000 7f3fffe00640 1 -- 172.21.15.143:0/1081516421 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2657732558 0 0) 0x7f40000cece0 con 0x7f400015c7f0 2024-01-16T15:01:20.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.523+0000 7f3fffe00640 1 -- 172.21.15.143:0/1081516421 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fe0003520 con 0x7f400015c7f0 2024-01-16T15:01:20.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.523+0000 7f3fffe00640 1 -- 172.21.15.143:0/1081516421 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2124308185 0 0) 0x7f3fe0003520 con 0x7f400015c7f0 2024-01-16T15:01:20.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.523+0000 7f3fffe00640 1 -- 172.21.15.143:0/1081516421 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40000cdfb0 msgr2=0x7f40000ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:20.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.523+0000 7f3fffe00640 1 --2- 172.21.15.143:0/1081516421 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40000cdfb0 0x7f40000ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:20.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.523+0000 7f3fffe00640 1 -- 172.21.15.143:0/1081516421 >> v2:172.21.15.204:3300/0 conn(0x7f40000cf450 msgr2=0x7f40000cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:20.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.523+0000 7f3fffe00640 1 --2- 172.21.15.143:0/1081516421 >> v2:172.21.15.204:3300/0 conn(0x7f40000cf450 0x7f40000cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:20.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.523+0000 7f3fffe00640 1 -- 172.21.15.143:0/1081516421 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f400006d040 con 0x7f400015c7f0 2024-01-16T15:01:20.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.523+0000 7f3fffe00640 1 -- 172.21.15.143:0/1081516421 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7f3ff0001670 con 0x7f400015c7f0 2024-01-16T15:01:20.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.523+0000 7f3fffe00640 1 -- 172.21.15.143:0/1081516421 >> v1:172.21.15.143:6789/0 conn(0x7f400015c7f0 legacy=0x7f400006db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:20.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.524+0000 7f3fffe00640 1 --2- 172.21.15.143:0/1081516421 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40000cdfb0 0x7f3fe0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:20.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.524+0000 7f3fffe00640 1 --2- 172.21.15.143:0/1081516421 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fe0003f40 0x7f3fe00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:20.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.524+0000 7f3fffe00640 1 --2- 172.21.15.143:0/1081516421 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fe0004d00 0x7f3fe0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:20.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.524+0000 7f3fffe00640 1 -- 172.21.15.143:0/1081516421 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40000cfe40 con 0x7f3fe0004d00 2024-01-16T15:01:20.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.524+0000 7f3fffe00640 1 -- 172.21.15.143:0/1081516421 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4000156010 con 0x7f3fe0003f40 2024-01-16T15:01:20.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.524+0000 7f3fffe00640 1 -- 172.21.15.143:0/1081516421 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe000ba30 con 0x7f40000cdfb0 2024-01-16T15:01:20.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.524+0000 7f4005000640 1 --2- 172.21.15.143:0/1081516421 >> v2:172.21.15.204:3300/0 conn(0x7f40000cf450 0x7f40000cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T15:01:20.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.524+0000 7f4006400640 1 --2- 172.21.15.143:0/1081516421 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fe0004d00 0x7f3fe0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:20.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.524+0000 7f4006400640 1 -- 172.21.15.143:0/1081516421 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40000cdfb0 msgr2=0x7f3fe0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:20.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.524+0000 7f4006400640 1 --2- 172.21.15.143:0/1081516421 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40000cdfb0 0x7f3fe0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:20.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.524+0000 7f4006400640 1 -- 172.21.15.143:0/1081516421 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fe0003f40 msgr2=0x7f3fe00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:20.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.524+0000 7f4006400640 1 --2- 172.21.15.143:0/1081516421 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fe0003f40 0x7f3fe00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:20.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.524+0000 7f4006400640 1 -- 172.21.15.143:0/1081516421 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe0003520 con 0x7f3fe0004d00 2024-01-16T15:01:20.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.525+0000 7f4006400640 1 --2- 172.21.15.143:0/1081516421 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fe0004d00 0x7f3fe0005180 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8009fc0 tx=0x7f3fe00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=638b8a882e3b51aa server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:20.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.526+0000 7f3fffe00640 1 -- 172.21.15.143:0/1081516421 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff802cc60 con 0x7f3fe0004d00 2024-01-16T15:01:20.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.526+0000 7f3fffe00640 1 -- 172.21.15.143:0/1081516421 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff800f040 con 0x7f3fe0004d00 2024-01-16T15:01:20.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.526+0000 7f3fffe00640 1 -- 172.21.15.143:0/1081516421 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8034440 con 0x7f3fe0004d00 2024-01-16T15:01:20.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.526+0000 7f4008400640 1 -- 172.21.15.143:0/1081516421 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fe0004d00 msgr2=0x7f3fe0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:20.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.526+0000 7f4008400640 1 --2- 172.21.15.143:0/1081516421 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fe0004d00 0x7f3fe0005180 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8009fc0 tx=0x7f3fe00057f0 comp rx=0 tx=0).stop 2024-01-16T15:01:20.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.526+0000 7f4006400640 1 -- 172.21.15.143:0/1081516421 reap_dead start 2024-01-16T15:01:20.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.526+0000 7f4008400640 1 -- 172.21.15.143:0/1081516421 shutdown_connections 2024-01-16T15:01:20.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.526+0000 7f4008400640 1 -- 172.21.15.143:0/1081516421 >> 172.21.15.143:0/1081516421 conn(0x7f400006a920 msgr2=0x7f400006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:20.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.526+0000 7f4008400640 1 -- 172.21.15.143:0/1081516421 shutdown_connections 2024-01-16T15:01:20.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.526+0000 7f4008400640 1 -- 172.21.15.143:0/1081516421 wait complete. 2024-01-16T15:01:20.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.528+0000 7f4008400640 1 Processor -- start 2024-01-16T15:01:20.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.528+0000 7f4008400640 1 -- start start 2024-01-16T15:01:20.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.529+0000 7f4008400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f400014b5b0 0x7f40000addc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:20.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.529+0000 7f4008400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40000ae300 0x7f40000af560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:20.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.529+0000 7f4008400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40000afaa0 0x7f40000b20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:20.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.529+0000 7f4008400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40000ad000 con 0x7f400014b5b0 2024-01-16T15:01:20.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.529+0000 7f4008400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40000aff00 con 0x7f40000ae300 2024-01-16T15:01:20.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.529+0000 7f4008400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40000ac380 con 0x7f40000afaa0 2024-01-16T15:01:20.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.529+0000 7f4005a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f400014b5b0 0x7f40000addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:20.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.529+0000 7f4005a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f400014b5b0 0x7f40000addc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59118/0 (socket says 172.21.15.143:59118) 2024-01-16T15:01:20.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.529+0000 7f4005a00640 1 -- 172.21.15.143:0/4136171648 learned_addr learned my addr 172.21.15.143:0/4136171648 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:20.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.530+0000 7f4006400640 1 --2- 172.21.15.143:0/4136171648 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40000afaa0 0x7f40000b20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:20.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.530+0000 7f4005000640 1 --2- 172.21.15.143:0/4136171648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40000ae300 0x7f40000af560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:20.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.530+0000 7f4005a00640 1 -- 172.21.15.143:0/4136171648 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40000afaa0 msgr2=0x7f40000b20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:20.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.530+0000 7f4005a00640 1 --2- 172.21.15.143:0/4136171648 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40000afaa0 0x7f40000b20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:20.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.530+0000 7f4005a00640 1 -- 172.21.15.143:0/4136171648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40000ae300 msgr2=0x7f40000af560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:20.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.530+0000 7f4005a00640 1 --2- 172.21.15.143:0/4136171648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40000ae300 0x7f40000af560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:20.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.530+0000 7f4005a00640 1 -- 172.21.15.143:0/4136171648 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40000ac660 con 0x7f400014b5b0 2024-01-16T15:01:20.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.530+0000 7f4005a00640 1 --2- 172.21.15.143:0/4136171648 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f400014b5b0 0x7f40000addc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000ca70 tx=0x7f3ff000cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:20.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.530+0000 7f3ffd600640 1 -- 172.21.15.143:0/4136171648 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0004cb0 con 0x7f400014b5b0 2024-01-16T15:01:20.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.530+0000 7f3ffd600640 1 -- 172.21.15.143:0/4136171648 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff0004e10 con 0x7f400014b5b0 2024-01-16T15:01:20.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.530+0000 7f3ffd600640 1 -- 172.21.15.143:0/4136171648 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0007310 con 0x7f400014b5b0 2024-01-16T15:01:20.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.530+0000 7f4008400640 1 -- 172.21.15.143:0/4136171648 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40000ac940 con 0x7f400014b5b0 2024-01-16T15:01:20.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.530+0000 7f4008400640 1 -- 172.21.15.143:0/4136171648 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40000abfc0 con 0x7f400014b5b0 2024-01-16T15:01:20.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.532+0000 7f4008400640 1 -- 172.21.15.143:0/4136171648 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4000067470 con 0x7f400014b5b0 2024-01-16T15:01:20.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.532+0000 7f3ffd600640 1 -- 172.21.15.143:0/4136171648 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f3ff0007470 con 0x7f400014b5b0 2024-01-16T15:01:20.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.533+0000 7f3ffd600640 1 --2- 172.21.15.143:0/4136171648 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f3fd806b2c0 0x7f3fd806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:20.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.533+0000 7f3ffd600640 1 -- 172.21.15.143:0/4136171648 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ff0089170 con 0x7f400014b5b0 2024-01-16T15:01:20.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.534+0000 7f4005000640 1 --2- 172.21.15.143:0/4136171648 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f3fd806b2c0 0x7f3fd806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:20.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.534+0000 7f4005000640 1 --2- 172.21.15.143:0/4136171648 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f3fd806b2c0 0x7f3fd806d780 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f400009d310 tx=0x7f3fe8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:20.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.538+0000 7f3ffd600640 1 -- 172.21.15.143:0/4136171648 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3ff0053f40 con 0x7f400014b5b0 2024-01-16T15:01:20.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.715+0000 7f4008400640 1 -- 172.21.15.143:0/4136171648 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40000cd730 con 0x7f400014b5b0 2024-01-16T15:01:20.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.717+0000 7f3ffd600640 1 -- 172.21.15.143:0/4136171648 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16642 (secure 0 0 0) 0x7f3ff0057560 con 0x7f400014b5b0 2024-01-16T15:01:20.697 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:01:20.698 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":33,"fsid":"c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0","created":"2024-01-16T14:59:38.532601+0000","modified":"2024-01-16T15:01:19.289691+0000","last_up_change":"2024-01-16T14:59:50.983699+0000","last_in_change":"2024-01-16T14:59:42.056297+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:59:51.727720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-01-16T15:01:17.024625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-01-16T15:01:18.326532+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f4c8891d-ab29-472a-907d-8446f20f7012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6801","nonce":1569293626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6805","nonce":1569293626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6814","nonce":1569293626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6809","nonce":1569293626}]},"public_addr":"172.21.15.143:6801/1569293626","cluster_addr":"172.21.15.143:6805/1569293626","heartbeat_back_addr":"172.21.15.143:6814/1569293626","heartbeat_front_addr":"172.21.15.143:6809/1569293626","state":["exists","up"]},{"osd":1,"uuid":"c4a2788d-8b89-4992-82b2-7f4b9debd509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6804","nonce":1276841168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6808","nonce":1276841168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6813","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6815","nonce":1276841168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6811","nonce":1276841168}]},"public_addr":"172.21.15.143:6804/1276841168","cluster_addr":"172.21.15.143:6808/1276841168","heartbeat_back_addr":"172.21.15.143:6815/1276841168","heartbeat_front_addr":"172.21.15.143:6811/1276841168","state":["exists","up"]},{"osd":2,"uuid":"64645c1d-31b2-4794-b0f2-7ccf8df30393","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6817","nonce":1217670170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6819","nonce":1217670170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6823","nonce":1217670170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6821","nonce":1217670170}]},"public_addr":"172.21.15.143:6817/1217670170","cluster_addr":"172.21.15.143:6819/1217670170","heartbeat_back_addr":"172.21.15.143:6823/1217670170","heartbeat_front_addr":"172.21.15.143:6821/1217670170","state":["exists","up"]},{"osd":3,"uuid":"73674129-a84e-4d31-8247-7d0179ae6df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6825","nonce":3414188012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6827","nonce":3414188012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6831","nonce":3414188012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6829","nonce":3414188012}]},"public_addr":"172.21.15.143:6825/3414188012","cluster_addr":"172.21.15.143:6827/3414188012","heartbeat_back_addr":"172.21.15.143:6831/3414188012","heartbeat_front_addr":"172.21.15.143:6829/3414188012","state":["exists","up"]},{"osd":4,"uuid":"cea28c9b-2f1c-4087-b415-9960789131d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6811","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6812","nonce":1070255716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6816","nonce":1070255716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6823","nonce":1070255716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6820","nonce":1070255716}]},"public_addr":"172.21.15.204:6812/1070255716","cluster_addr":"172.21.15.204:6816/1070255716","heartbeat_back_addr":"172.21.15.204:6823/1070255716","heartbeat_front_addr":"172.21.15.204:6820/1070255716","state":["exists","up"]},{"osd":5,"uuid":"33b63560-5574-4e78-983f-b398d67a2cb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6809","nonce":3605417837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6813","nonce":3605417837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6819","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6821","nonce":3605417837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6817","nonce":3605417837}]},"public_addr":"172.21.15.204:6809/3605417837","cluster_addr":"172.21.15.204:6813/3605417837","heartbeat_back_addr":"172.21.15.204:6821/3605417837","heartbeat_front_addr":"172.21.15.204:6817/3605417837","state":["exists","up"]},{"osd":6,"uuid":"df397427-8940-4eee-9c79-eece669d735d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6801","nonce":1085000534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6803","nonce":1085000534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6808","nonce":1085000534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6805","nonce":1085000534}]},"public_addr":"172.21.15.204:6801/1085000534","cluster_addr":"172.21.15.204:6803/1085000534","heartbeat_back_addr":"172.21.15.204:6808/1085000534","heartbeat_front_addr":"172.21.15.204:6805/1085000534","state":["exists","up"]},{"osd":7,"uuid":"a2e5a15c-8312-405a-ac02-f7e1fc9df2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6825","nonce":1337220609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6827","nonce":1337220609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6831","nonce":1337220609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6829","nonce":1337220609}]},"public_addr":"172.21.15.204:6825/1337220609","cluster_addr":"172.21.15.204:6827/1337220609","heartbeat_back_addr":"172.21.15.204:6831/1337220609","heartbeat_front_addr":"172.21.15.204:6829/1337220609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:49.984507+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.395306+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.632998+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.373627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.342920+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.438366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.331926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.406657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T15:01:20.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.720+0000 7f4008400640 1 -- 172.21.15.143:0/4136171648 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f3fd806b2c0 msgr2=0x7f3fd806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:20.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.720+0000 7f4008400640 1 --2- 172.21.15.143:0/4136171648 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f3fd806b2c0 0x7f3fd806d780 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f400009d310 tx=0x7f3fe8008040 comp rx=0 tx=0).stop 2024-01-16T15:01:20.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.720+0000 7f4008400640 1 -- 172.21.15.143:0/4136171648 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f400014b5b0 msgr2=0x7f40000addc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:20.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.720+0000 7f4008400640 1 --2- 172.21.15.143:0/4136171648 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f400014b5b0 0x7f40000addc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000ca70 tx=0x7f3ff000cf30 comp rx=0 tx=0).stop 2024-01-16T15:01:20.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.720+0000 7f4008400640 1 -- 172.21.15.143:0/4136171648 shutdown_connections 2024-01-16T15:01:20.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.720+0000 7f4008400640 1 --2- 172.21.15.143:0/4136171648 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f3fd806b2c0 0x7f3fd806d780 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:20.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.720+0000 7f4008400640 1 --2- 172.21.15.143:0/4136171648 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40000afaa0 0x7f40000b20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:20.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.720+0000 7f4008400640 1 --2- 172.21.15.143:0/4136171648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40000ae300 0x7f40000af560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:20.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.720+0000 7f4008400640 1 --2- 172.21.15.143:0/4136171648 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f400014b5b0 0x7f40000addc0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:20.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.720+0000 7f4008400640 1 -- 172.21.15.143:0/4136171648 >> 172.21.15.143:0/4136171648 conn(0x7f400006a920 msgr2=0x7f400015def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:20.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.720+0000 7f4008400640 1 -- 172.21.15.143:0/4136171648 shutdown_connections 2024-01-16T15:01:20.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.720+0000 7f4008400640 1 -- 172.21.15.143:0/4136171648 wait complete. 2024-01-16T15:01:20.717 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-16T15:01:20.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.942+0000 7f94a2a00640 1 Processor -- start 2024-01-16T15:01:20.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.943+0000 7f94a2a00640 1 -- start start 2024-01-16T15:01:20.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.944+0000 7f94a2a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f949c0cc0f0 0x7f949c0c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:20.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.944+0000 7f94a2a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f949c0c57e0 0x7f949c06c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:20.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.944+0000 7f94a2a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f949c15eea0 con 0x7f949c15b440 2024-01-16T15:01:20.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.944+0000 7f94a2a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f949c15b820 con 0x7f949c0cc0f0 2024-01-16T15:01:20.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.944+0000 7f94a2a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f949c0abf30 con 0x7f949c0c57e0 2024-01-16T15:01:20.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.944+0000 7f949be00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f949c15b440 0x7f949c0cb9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57790/0 (socket says 172.21.15.143:57790) 2024-01-16T15:01:20.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.944+0000 7f949be00640 1 -- 172.21.15.143:0/2939152675 learned_addr learned my addr 172.21.15.143:0/2939152675 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:01:20.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.945+0000 7f94a0a00640 1 --2- 172.21.15.143:0/2939152675 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f949c0c57e0 0x7f949c06c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:20.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.945+0000 7f949b400640 1 --2- 172.21.15.143:0/2939152675 >> v2:172.21.15.204:3300/0 conn(0x7f949c0cc0f0 0x7f949c0c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:20.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.945+0000 7f949aa00640 1 -- 172.21.15.143:0/2939152675 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1079075042 0 0) 0x7f949c15eea0 con 0x7f949c15b440 2024-01-16T15:01:20.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.945+0000 7f949aa00640 1 -- 172.21.15.143:0/2939152675 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f947c003520 con 0x7f949c15b440 2024-01-16T15:01:20.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.945+0000 7f949aa00640 1 -- 172.21.15.143:0/2939152675 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1147289177 0 0) 0x7f947c003520 con 0x7f949c15b440 2024-01-16T15:01:20.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.945+0000 7f949aa00640 1 -- 172.21.15.143:0/2939152675 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f949c0c57e0 msgr2=0x7f949c06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:20.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.945+0000 7f949aa00640 1 --2- 172.21.15.143:0/2939152675 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f949c0c57e0 0x7f949c06c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:20.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.945+0000 7f949aa00640 1 -- 172.21.15.143:0/2939152675 >> v2:172.21.15.204:3300/0 conn(0x7f949c0cc0f0 msgr2=0x7f949c0c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:20.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.945+0000 7f949aa00640 1 --2- 172.21.15.143:0/2939152675 >> v2:172.21.15.204:3300/0 conn(0x7f949c0cc0f0 0x7f949c0c52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:20.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.945+0000 7f949aa00640 1 -- 172.21.15.143:0/2939152675 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f949c15f0f0 con 0x7f949c15b440 2024-01-16T15:01:20.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.945+0000 7f949aa00640 1 -- 172.21.15.143:0/2939152675 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7f948c001670 con 0x7f949c15b440 2024-01-16T15:01:20.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.945+0000 7f949aa00640 1 -- 172.21.15.143:0/2939152675 >> v1:172.21.15.143:6789/0 conn(0x7f949c15b440 legacy=0x7f949c0cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:20.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.945+0000 7f949aa00640 1 --2- 172.21.15.143:0/2939152675 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f949c0c57e0 0x7f947c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:20.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.945+0000 7f949aa00640 1 --2- 172.21.15.143:0/2939152675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f947c003ef0 0x7f947c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:20.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.945+0000 7f949aa00640 1 --2- 172.21.15.143:0/2939152675 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f947c004d00 0x7f947c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:20.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.945+0000 7f949aa00640 1 -- 172.21.15.143:0/2939152675 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949c15b820 con 0x7f947c004d00 2024-01-16T15:01:20.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.945+0000 7f949aa00640 1 -- 172.21.15.143:0/2939152675 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949c0abf30 con 0x7f947c003ef0 2024-01-16T15:01:20.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.945+0000 7f949aa00640 1 -- 172.21.15.143:0/2939152675 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f947c00ba80 con 0x7f949c0c57e0 2024-01-16T15:01:20.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.946+0000 7f949b400640 1 --2- 172.21.15.143:0/2939152675 >> v2:172.21.15.204:3300/0 conn(0x7f949c0cc0f0 0x7f949c0c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T15:01:20.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.946+0000 7f94a0a00640 1 --2- 172.21.15.143:0/2939152675 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f947c004d00 0x7f947c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:20.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.946+0000 7f949be00640 1 --2- 172.21.15.143:0/2939152675 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f949c0c57e0 0x7f947c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:20.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.946+0000 7f949b400640 1 --2- 172.21.15.143:0/2939152675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f947c003ef0 0x7f947c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:20.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.946+0000 7f94a0a00640 1 -- 172.21.15.143:0/2939152675 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f949c0c57e0 msgr2=0x7f947c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:20.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.946+0000 7f94a0a00640 1 --2- 172.21.15.143:0/2939152675 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f949c0c57e0 0x7f947c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:20.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.946+0000 7f94a0a00640 1 -- 172.21.15.143:0/2939152675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f947c003ef0 msgr2=0x7f947c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:20.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.946+0000 7f94a0a00640 1 --2- 172.21.15.143:0/2939152675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f947c003ef0 0x7f947c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:20.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.946+0000 7f94a0a00640 1 -- 172.21.15.143:0/2939152675 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f947c003520 con 0x7f947c004d00 2024-01-16T15:01:20.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.946+0000 7f949b400640 1 --2- 172.21.15.143:0/2939152675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f947c003ef0 0x7f947c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:01:20.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.946+0000 7f949be00640 1 --2- 172.21.15.143:0/2939152675 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f949c0c57e0 0x7f947c003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:01:20.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.947+0000 7f94a0a00640 1 --2- 172.21.15.143:0/2939152675 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f947c004d00 0x7f947c005180 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f949400e170 tx=0x7f947c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=399253f08541135f server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:20.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.948+0000 7f949aa00640 1 -- 172.21.15.143:0/2939152675 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949402cbf0 con 0x7f947c004d00 2024-01-16T15:01:20.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.948+0000 7f949aa00640 1 -- 172.21.15.143:0/2939152675 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f949400f040 con 0x7f947c004d00 2024-01-16T15:01:20.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.948+0000 7f949aa00640 1 -- 172.21.15.143:0/2939152675 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9494034440 con 0x7f947c004d00 2024-01-16T15:01:20.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.948+0000 7f94a2a00640 1 -- 172.21.15.143:0/2939152675 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f947c004d00 msgr2=0x7f947c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:20.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.948+0000 7f94a2a00640 1 --2- 172.21.15.143:0/2939152675 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f947c004d00 0x7f947c005180 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f949400e170 tx=0x7f947c0057f0 comp rx=0 tx=0).stop 2024-01-16T15:01:20.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.948+0000 7f94a0a00640 1 -- 172.21.15.143:0/2939152675 reap_dead start 2024-01-16T15:01:20.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.948+0000 7f94a2a00640 1 -- 172.21.15.143:0/2939152675 shutdown_connections 2024-01-16T15:01:20.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.948+0000 7f94a2a00640 1 -- 172.21.15.143:0/2939152675 >> 172.21.15.143:0/2939152675 conn(0x7f949c0b0c50 msgr2=0x7f949c0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:20.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.948+0000 7f94a2a00640 1 -- 172.21.15.143:0/2939152675 shutdown_connections 2024-01-16T15:01:20.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.948+0000 7f94a2a00640 1 -- 172.21.15.143:0/2939152675 wait complete. 2024-01-16T15:01:20.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.950+0000 7f94a2a00640 1 Processor -- start 2024-01-16T15:01:20.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.950+0000 7f94a2a00640 1 -- start start 2024-01-16T15:01:20.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.951+0000 7f94a2a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f947c004d00 0x7f949c156050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:20.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.951+0000 7f94a2a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f949c0cdc20 0x7f949c0ce080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:20.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.951+0000 7f94a2a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f949c0b20b0 0x7f949c0b2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:20.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.951+0000 7f94a2a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949c0ce5c0 con 0x7f947c004d00 2024-01-16T15:01:20.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.951+0000 7f94a2a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949c15d150 con 0x7f949c0b20b0 2024-01-16T15:01:20.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.951+0000 7f94a2a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f949c156590 con 0x7f949c0cdc20 2024-01-16T15:01:20.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.952+0000 7f949be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f947c004d00 0x7f949c156050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:20.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.952+0000 7f949be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f947c004d00 0x7f949c156050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59144/0 (socket says 172.21.15.143:59144) 2024-01-16T15:01:20.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.952+0000 7f949be00640 1 -- 172.21.15.143:0/1421870922 learned_addr learned my addr 172.21.15.143:0/1421870922 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:20.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.952+0000 7f949b400640 1 --2- 172.21.15.143:0/1421870922 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f949c0cdc20 0x7f949c0ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:20.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.952+0000 7f94a0a00640 1 --2- 172.21.15.143:0/1421870922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f949c0b20b0 0x7f949c0b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:20.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.952+0000 7f949be00640 1 -- 172.21.15.143:0/1421870922 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f949c0cdc20 msgr2=0x7f949c0ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:20.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.952+0000 7f949be00640 1 --2- 172.21.15.143:0/1421870922 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f949c0cdc20 0x7f949c0ce080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:20.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.952+0000 7f949be00640 1 -- 172.21.15.143:0/1421870922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f949c0b20b0 msgr2=0x7f949c0b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:20.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.952+0000 7f949be00640 1 --2- 172.21.15.143:0/1421870922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f949c0b20b0 0x7f949c0b2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:20.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.952+0000 7f949be00640 1 -- 172.21.15.143:0/1421870922 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f949c0b2de0 con 0x7f947c004d00 2024-01-16T15:01:20.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.952+0000 7f949be00640 1 --2- 172.21.15.143:0/1421870922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f947c004d00 0x7f949c156050 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f948c00d8b0 tx=0x7f948c00dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:20.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.953+0000 7f9493e00640 1 -- 172.21.15.143:0/1421870922 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c0074e0 con 0x7f947c004d00 2024-01-16T15:01:20.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.953+0000 7f9493e00640 1 -- 172.21.15.143:0/1421870922 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948c007640 con 0x7f947c004d00 2024-01-16T15:01:20.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.953+0000 7f94a2a00640 1 -- 172.21.15.143:0/1421870922 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f949c0b30c0 con 0x7f947c004d00 2024-01-16T15:01:20.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.953+0000 7f9493e00640 1 -- 172.21.15.143:0/1421870922 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c0049e0 con 0x7f947c004d00 2024-01-16T15:01:20.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.953+0000 7f94a2a00640 1 -- 172.21.15.143:0/1421870922 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f949c0b2a70 con 0x7f947c004d00 2024-01-16T15:01:20.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.954+0000 7f94a2a00640 1 -- 172.21.15.143:0/1421870922 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f949c0cbf10 con 0x7f947c004d00 2024-01-16T15:01:20.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.955+0000 7f9493e00640 1 -- 172.21.15.143:0/1421870922 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f948c004b40 con 0x7f947c004d00 2024-01-16T15:01:20.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.956+0000 7f9493e00640 1 --2- 172.21.15.143:0/1421870922 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f947406b2c0 0x7f947406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:20.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.957+0000 7f9493e00640 1 -- 172.21.15.143:0/1421870922 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f948c089950 con 0x7f947c004d00 2024-01-16T15:01:20.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.957+0000 7f949b400640 1 --2- 172.21.15.143:0/1421870922 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f947406b2c0 0x7f947406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:20.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.957+0000 7f949b400640 1 --2- 172.21.15.143:0/1421870922 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f947406b2c0 0x7f947406d780 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f949c09c600 tx=0x7f9484008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:20.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:20.961+0000 7f9493e00640 1 -- 172.21.15.143:0/1421870922 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f948c0547c0 con 0x7f947c004d00 2024-01-16T15:01:21.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:21.182+0000 7f94a2a00640 1 -- 172.21.15.143:0/1421870922 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f949c04e8b0 con 0x7f947c004d00 2024-01-16T15:01:21.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:21.908+0000 7f9493e00640 1 -- 172.21.15.143:0/1421870922 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7f948c019020 con 0x7f947c004d00 2024-01-16T15:01:21.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:21.911+0000 7f94a2a00640 1 -- 172.21.15.143:0/1421870922 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f947406b2c0 msgr2=0x7f947406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:21.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:21.911+0000 7f94a2a00640 1 --2- 172.21.15.143:0/1421870922 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f947406b2c0 0x7f947406d780 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f949c09c600 tx=0x7f9484008040 comp rx=0 tx=0).stop 2024-01-16T15:01:21.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:21.911+0000 7f94a2a00640 1 -- 172.21.15.143:0/1421870922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f947c004d00 msgr2=0x7f949c156050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:21.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:21.911+0000 7f94a2a00640 1 --2- 172.21.15.143:0/1421870922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f947c004d00 0x7f949c156050 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f948c00d8b0 tx=0x7f948c00dd70 comp rx=0 tx=0).stop 2024-01-16T15:01:21.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:21.912+0000 7f94a2a00640 1 -- 172.21.15.143:0/1421870922 shutdown_connections 2024-01-16T15:01:21.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:21.912+0000 7f94a2a00640 1 --2- 172.21.15.143:0/1421870922 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f947406b2c0 0x7f947406d780 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:21.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:21.912+0000 7f94a2a00640 1 --2- 172.21.15.143:0/1421870922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f949c0b20b0 0x7f949c0b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:21.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:21.912+0000 7f94a2a00640 1 --2- 172.21.15.143:0/1421870922 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f949c0cdc20 0x7f949c0ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:21.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:21.912+0000 7f94a2a00640 1 --2- 172.21.15.143:0/1421870922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f947c004d00 0x7f949c156050 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:21.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:21.912+0000 7f94a2a00640 1 -- 172.21.15.143:0/1421870922 >> 172.21.15.143:0/1421870922 conn(0x7f949c0b0c50 msgr2=0x7f949c071950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:21.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:21.912+0000 7f94a2a00640 1 -- 172.21.15.143:0/1421870922 shutdown_connections 2024-01-16T15:01:21.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:21.912+0000 7f94a2a00640 1 -- 172.21.15.143:0/1421870922 wait complete. 2024-01-16T15:01:21.913 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-01-16T15:01:21.913 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-16T15:01:22.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.142+0000 7f7598000640 1 Processor -- start 2024-01-16T15:01:22.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.142+0000 7f7598000640 1 -- start start 2024-01-16T15:01:22.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.143+0000 7f7598000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f759015b440 0x7f75900cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:22.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.144+0000 7f7598000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75900cbf20 0x7f75900c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:22.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.144+0000 7f7598000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f759015eea0 con 0x7f75900c57e0 2024-01-16T15:01:22.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.144+0000 7f7598000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75900cc3d0 con 0x7f759015b440 2024-01-16T15:01:22.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.144+0000 7f7598000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f759015b820 con 0x7f75900cbf20 2024-01-16T15:01:22.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.144+0000 7f7595600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f759015b440 0x7f75900cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:22.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.144+0000 7f7595600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f759015b440 0x7f75900cb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59150/0 (socket says 172.21.15.143:59150) 2024-01-16T15:01:22.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.144+0000 7f7595600640 1 -- 172.21.15.143:0/2715971062 learned_addr learned my addr 172.21.15.143:0/2715971062 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:22.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.144+0000 7f7595600640 1 -- 172.21.15.143:0/2715971062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75900cbf20 msgr2=0x7f75900c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:22.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.144+0000 7f7595600640 1 --2- 172.21.15.143:0/2715971062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75900cbf20 0x7f75900c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:22.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.144+0000 7f7595600640 1 -- 172.21.15.143:0/2715971062 >> v1:172.21.15.204:6789/0 conn(0x7f75900c57e0 legacy=0x7f75900c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:22.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.144+0000 7f7595600640 1 -- 172.21.15.143:0/2715971062 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f759006d600 con 0x7f759015b440 2024-01-16T15:01:22.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.145+0000 7f7595600640 1 --2- 172.21.15.143:0/2715971062 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f759015b440 0x7f75900cb9e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7580009870 tx=0x7f758002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5dca22878c460f1 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:22.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.146+0000 7f758fe00640 1 -- 172.21.15.143:0/2715971062 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75800317c0 con 0x7f759015b440 2024-01-16T15:01:22.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.146+0000 7f758fe00640 1 -- 172.21.15.143:0/2715971062 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7580031920 con 0x7f759015b440 2024-01-16T15:01:22.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.146+0000 7f758fe00640 1 -- 172.21.15.143:0/2715971062 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7580031cb0 con 0x7f759015b440 2024-01-16T15:01:22.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.146+0000 7f7598000640 1 -- 172.21.15.143:0/2715971062 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f759015b440 msgr2=0x7f75900cb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:22.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.146+0000 7f7598000640 1 --2- 172.21.15.143:0/2715971062 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f759015b440 0x7f75900cb9e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7580009870 tx=0x7f758002b860 comp rx=0 tx=0).stop 2024-01-16T15:01:22.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.146+0000 7f7598000640 1 -- 172.21.15.143:0/2715971062 shutdown_connections 2024-01-16T15:01:22.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.146+0000 7f7598000640 1 --2- 172.21.15.143:0/2715971062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75900cbf20 0x7f75900c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:22.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.146+0000 7f7598000640 1 --2- 172.21.15.143:0/2715971062 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f759015b440 0x7f75900cb9e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:22.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.146+0000 7f7598000640 1 -- 172.21.15.143:0/2715971062 >> 172.21.15.143:0/2715971062 conn(0x7f75900b0c50 msgr2=0x7f75900b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:22.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.146+0000 7f7598000640 1 -- 172.21.15.143:0/2715971062 shutdown_connections 2024-01-16T15:01:22.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.146+0000 7f7598000640 1 -- 172.21.15.143:0/2715971062 wait complete. 2024-01-16T15:01:22.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.148+0000 7f7598000640 1 Processor -- start 2024-01-16T15:01:22.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.148+0000 7f7598000640 1 -- start start 2024-01-16T15:01:22.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.149+0000 7f7598000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75900c57e0 0x7f759009a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:22.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.149+0000 7f7598000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75900cbf20 0x7f759009b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:22.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.149+0000 7f7598000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f759015b440 0x7f7590155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:22.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.149+0000 7f7598000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759009a090 con 0x7f759015b440 2024-01-16T15:01:22.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.149+0000 7f7598000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759009a9f0 con 0x7f75900c57e0 2024-01-16T15:01:22.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.149+0000 7f7598000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f759009bde0 con 0x7f75900cbf20 2024-01-16T15:01:22.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.149+0000 7f7596000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f759015b440 0x7f7590155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:22.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.150+0000 7f7596000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f759015b440 0x7f7590155db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59152/0 (socket says 172.21.15.143:59152) 2024-01-16T15:01:22.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.150+0000 7f7596000640 1 -- 172.21.15.143:0/2331435622 learned_addr learned my addr 172.21.15.143:0/2331435622 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:22.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.150+0000 7f7595600640 1 --2- 172.21.15.143:0/2331435622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75900c57e0 0x7f759009a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:22.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.150+0000 7f7594c00640 1 --2- 172.21.15.143:0/2331435622 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75900cbf20 0x7f759009b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:22.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.150+0000 7f7596000640 1 -- 172.21.15.143:0/2331435622 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75900cbf20 msgr2=0x7f759009b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:22.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.150+0000 7f7596000640 1 --2- 172.21.15.143:0/2331435622 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75900cbf20 0x7f759009b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:22.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.150+0000 7f7596000640 1 -- 172.21.15.143:0/2331435622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75900c57e0 msgr2=0x7f759009a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:22.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.150+0000 7f7596000640 1 --2- 172.21.15.143:0/2331435622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75900c57e0 0x7f759009a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:22.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.150+0000 7f7596000640 1 -- 172.21.15.143:0/2331435622 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75800094d0 con 0x7f759015b440 2024-01-16T15:01:22.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.150+0000 7f7594c00640 1 --2- 172.21.15.143:0/2331435622 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75900cbf20 0x7f759009b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:01:22.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.150+0000 7f7596000640 1 --2- 172.21.15.143:0/2331435622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f759015b440 0x7f7590155db0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f758800c780 tx=0x7f758800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:22.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.150+0000 7f758d600640 1 -- 172.21.15.143:0/2331435622 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7588003d80 con 0x7f759015b440 2024-01-16T15:01:22.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.151+0000 7f758d600640 1 -- 172.21.15.143:0/2331435622 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f758800ce90 con 0x7f759015b440 2024-01-16T15:01:22.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.151+0000 7f758d600640 1 -- 172.21.15.143:0/2331435622 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75880052b0 con 0x7f759015b440 2024-01-16T15:01:22.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.151+0000 7f7598000640 1 -- 172.21.15.143:0/2331435622 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f759009c0c0 con 0x7f759015b440 2024-01-16T15:01:22.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.151+0000 7f7598000640 1 -- 172.21.15.143:0/2331435622 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7590099cb0 con 0x7f759015b440 2024-01-16T15:01:22.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.152+0000 7f7598000640 1 -- 172.21.15.143:0/2331435622 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f759009a1d0 con 0x7f759015b440 2024-01-16T15:01:22.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.153+0000 7f758d600640 1 -- 172.21.15.143:0/2331435622 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f7588005410 con 0x7f759015b440 2024-01-16T15:01:22.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.154+0000 7f758d600640 1 --2- 172.21.15.143:0/2331435622 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f756806b2c0 0x7f756806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:22.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.154+0000 7f758d600640 1 -- 172.21.15.143:0/2331435622 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7588089120 con 0x7f759015b440 2024-01-16T15:01:22.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.154+0000 7f7595600640 1 --2- 172.21.15.143:0/2331435622 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f756806b2c0 0x7f756806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:22.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.155+0000 7f7595600640 1 --2- 172.21.15.143:0/2331435622 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f756806b2c0 0x7f756806d780 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7580036000 tx=0x7f758003d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:22.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.158+0000 7f758d600640 1 -- 172.21.15.143:0/2331435622 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f759009a1d0 con 0x7f759015b440 2024-01-16T15:01:22.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.367+0000 7f7598000640 1 -- 172.21.15.143:0/2331435622 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f759009a1d0 con 0x7f759015b440 2024-01-16T15:01:22.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.911+0000 7f758d600640 1 -- 172.21.15.143:0/2331435622 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7f759009a1d0 con 0x7f759015b440 2024-01-16T15:01:22.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.915+0000 7f7598000640 1 -- 172.21.15.143:0/2331435622 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f756806b2c0 msgr2=0x7f756806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:22.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.915+0000 7f7598000640 1 --2- 172.21.15.143:0/2331435622 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f756806b2c0 0x7f756806d780 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7580036000 tx=0x7f758003d040 comp rx=0 tx=0).stop 2024-01-16T15:01:22.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.915+0000 7f7598000640 1 -- 172.21.15.143:0/2331435622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f759015b440 msgr2=0x7f7590155db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:22.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.915+0000 7f7598000640 1 --2- 172.21.15.143:0/2331435622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f759015b440 0x7f7590155db0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f758800c780 tx=0x7f758800cc40 comp rx=0 tx=0).stop 2024-01-16T15:01:22.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.916+0000 7f7598000640 1 -- 172.21.15.143:0/2331435622 shutdown_connections 2024-01-16T15:01:22.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.916+0000 7f7598000640 1 --2- 172.21.15.143:0/2331435622 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f756806b2c0 0x7f756806d780 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:22.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.916+0000 7f7598000640 1 --2- 172.21.15.143:0/2331435622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f759015b440 0x7f7590155db0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:22.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.916+0000 7f7598000640 1 --2- 172.21.15.143:0/2331435622 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75900cbf20 0x7f759009b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:22.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.916+0000 7f7598000640 1 --2- 172.21.15.143:0/2331435622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75900c57e0 0x7f759009a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:22.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.916+0000 7f7598000640 1 -- 172.21.15.143:0/2331435622 >> 172.21.15.143:0/2331435622 conn(0x7f75900b0c50 msgr2=0x7f75900b04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:22.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.916+0000 7f7598000640 1 -- 172.21.15.143:0/2331435622 shutdown_connections 2024-01-16T15:01:22.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:22.916+0000 7f7598000640 1 -- 172.21.15.143:0/2331435622 wait complete. 2024-01-16T15:01:22.915 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-16T15:01:22.915 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-16T15:01:23.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.140+0000 7fe966600640 1 Processor -- start 2024-01-16T15:01:23.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.140+0000 7fe966600640 1 -- start start 2024-01-16T15:01:23.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.142+0000 7fe966600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe96015d4e0 0x7fe9600d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:23.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.142+0000 7fe966600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe9600d38e0 0x7fe9600cfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:23.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.142+0000 7fe966600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9600ce990 con 0x7fe9600d0070 2024-01-16T15:01:23.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.142+0000 7fe966600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9600d4110 con 0x7fe9600d38e0 2024-01-16T15:01:23.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.142+0000 7fe966600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9600d04e0 con 0x7fe96015d4e0 2024-01-16T15:01:23.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.142+0000 7fe95fe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe9600d38e0 0x7fe9600cfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:23.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.142+0000 7fe95fe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe9600d38e0 0x7fe9600cfb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59160/0 (socket says 172.21.15.143:59160) 2024-01-16T15:01:23.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.142+0000 7fe95fe00640 1 -- 172.21.15.143:0/855645039 learned_addr learned my addr 172.21.15.143:0/855645039 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:23.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.142+0000 7fe95fe00640 1 -- 172.21.15.143:0/855645039 >> v2:172.21.15.204:3300/0 conn(0x7fe96015d4e0 msgr2=0x7fe9600d1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:23.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.142+0000 7fe95fe00640 1 --2- 172.21.15.143:0/855645039 >> v2:172.21.15.204:3300/0 conn(0x7fe96015d4e0 0x7fe9600d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:23.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.142+0000 7fe95fe00640 1 -- 172.21.15.143:0/855645039 >> v1:172.21.15.204:6789/0 conn(0x7fe9600d0070 legacy=0x7fe9600d0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:23.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.142+0000 7fe95fe00640 1 -- 172.21.15.143:0/855645039 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9600cec70 con 0x7fe9600d38e0 2024-01-16T15:01:23.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.143+0000 7fe95fe00640 1 --2- 172.21.15.143:0/855645039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9600d38e0 0x7fe9600cfb30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe96008b800 tx=0x7fe94802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6081bda249b5283 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:23.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.144+0000 7fe95f400640 1 -- 172.21.15.143:0/855645039 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe948031870 con 0x7fe9600d38e0 2024-01-16T15:01:23.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.144+0000 7fe95f400640 1 -- 172.21.15.143:0/855645039 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9480319d0 con 0x7fe9600d38e0 2024-01-16T15:01:23.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.144+0000 7fe95f400640 1 -- 172.21.15.143:0/855645039 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe948031d60 con 0x7fe9600d38e0 2024-01-16T15:01:23.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.144+0000 7fe966600640 1 -- 172.21.15.143:0/855645039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9600d38e0 msgr2=0x7fe9600cfb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:23.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.144+0000 7fe966600640 1 --2- 172.21.15.143:0/855645039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9600d38e0 0x7fe9600cfb30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe96008b800 tx=0x7fe94802b910 comp rx=0 tx=0).stop 2024-01-16T15:01:23.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.145+0000 7fe966600640 1 -- 172.21.15.143:0/855645039 shutdown_connections 2024-01-16T15:01:23.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.145+0000 7fe966600640 1 --2- 172.21.15.143:0/855645039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9600d38e0 0x7fe9600cfb30 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:23.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.145+0000 7fe966600640 1 --2- 172.21.15.143:0/855645039 >> v2:172.21.15.204:3300/0 conn(0x7fe96015d4e0 0x7fe9600d1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:23.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.145+0000 7fe966600640 1 -- 172.21.15.143:0/855645039 >> 172.21.15.143:0/855645039 conn(0x7fe96009c510 msgr2=0x7fe9600d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:23.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.145+0000 7fe966600640 1 -- 172.21.15.143:0/855645039 shutdown_connections 2024-01-16T15:01:23.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.145+0000 7fe966600640 1 -- 172.21.15.143:0/855645039 wait complete. 2024-01-16T15:01:23.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.147+0000 7fe966600640 1 Processor -- start 2024-01-16T15:01:23.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.147+0000 7fe966600640 1 -- start start 2024-01-16T15:01:23.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.148+0000 7fe966600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9600d0070 0x7fe9600b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:23.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.148+0000 7fe966600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9600d38e0 0x7fe9600c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:23.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.148+0000 7fe966600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe96015d4e0 0x7fe9600b76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:23.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.148+0000 7fe966600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe960160090 con 0x7fe9600d0070 2024-01-16T15:01:23.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.148+0000 7fe966600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9600c9c80 con 0x7fe9600d38e0 2024-01-16T15:01:23.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.148+0000 7fe966600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9601604b0 con 0x7fe96015d4e0 2024-01-16T15:01:23.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.148+0000 7fe965200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9600d0070 0x7fe9600b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:23.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.148+0000 7fe965200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9600d0070 0x7fe9600b5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59174/0 (socket says 172.21.15.143:59174) 2024-01-16T15:01:23.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.148+0000 7fe965200640 1 -- 172.21.15.143:0/3864831184 learned_addr learned my addr 172.21.15.143:0/3864831184 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:23.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.149+0000 7fe965200640 1 -- 172.21.15.143:0/3864831184 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe96015d4e0 msgr2=0x7fe9600b76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:23.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.149+0000 7fe965200640 1 --2- 172.21.15.143:0/3864831184 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe96015d4e0 0x7fe9600b76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:23.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.149+0000 7fe95fe00640 1 --2- 172.21.15.143:0/3864831184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9600d38e0 0x7fe9600c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:23.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.149+0000 7fe965200640 1 -- 172.21.15.143:0/3864831184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9600d38e0 msgr2=0x7fe9600c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:23.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.149+0000 7fe965200640 1 --2- 172.21.15.143:0/3864831184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9600d38e0 0x7fe9600c9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:23.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.149+0000 7fe965200640 1 -- 172.21.15.143:0/3864831184 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9480094d0 con 0x7fe9600d0070 2024-01-16T15:01:23.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.149+0000 7fe965200640 1 --2- 172.21.15.143:0/3864831184 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9600d0070 0x7fe9600b5df0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe95000b410 tx=0x7fe95000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:23.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.149+0000 7fe95cc00640 1 -- 172.21.15.143:0/3864831184 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe950011830 con 0x7fe9600d0070 2024-01-16T15:01:23.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.149+0000 7fe95cc00640 1 -- 172.21.15.143:0/3864831184 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9500119b0 con 0x7fe9600d0070 2024-01-16T15:01:23.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.149+0000 7fe95cc00640 1 -- 172.21.15.143:0/3864831184 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe950011d60 con 0x7fe9600d0070 2024-01-16T15:01:23.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.149+0000 7fe966600640 1 -- 172.21.15.143:0/3864831184 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe96015f6f0 con 0x7fe9600d0070 2024-01-16T15:01:23.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.150+0000 7fe966600640 1 -- 172.21.15.143:0/3864831184 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe96015fcb0 con 0x7fe9600d0070 2024-01-16T15:01:23.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.151+0000 7fe966600640 1 -- 172.21.15.143:0/3864831184 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9600c7f30 con 0x7fe9600d0070 2024-01-16T15:01:23.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.151+0000 7fe95cc00640 1 -- 172.21.15.143:0/3864831184 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fe95001a400 con 0x7fe9600d0070 2024-01-16T15:01:23.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.152+0000 7fe95cc00640 1 --2- 172.21.15.143:0/3864831184 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe93806b2f0 0x7fe93806d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:23.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.152+0000 7fe95cc00640 1 -- 172.21.15.143:0/3864831184 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe950089d00 con 0x7fe9600d0070 2024-01-16T15:01:23.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.153+0000 7fe95fe00640 1 --2- 172.21.15.143:0/3864831184 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe93806b2f0 0x7fe93806d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:23.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.153+0000 7fe95fe00640 1 --2- 172.21.15.143:0/3864831184 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe93806b2f0 0x7fe93806d7b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe948036000 tx=0x7fe94803c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:23.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.157+0000 7fe95cc00640 1 -- 172.21.15.143:0/3864831184 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe950054b70 con 0x7fe9600d0070 2024-01-16T15:01:23.271 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:01:23.279 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:01:23.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.376+0000 7fe966600640 1 -- 172.21.15.143:0/3864831184 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe96004e840 con 0x7fe9600d0070 2024-01-16T15:01:23.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.377+0000 7fe95cc00640 1 -- 172.21.15.143:0/3864831184 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4919 (secure 0 0 0) 0x7fe950015020 con 0x7fe9600d0070 2024-01-16T15:01:23.356 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:01:23.357 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4732,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1793346439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1793346439},{"type":"v1","addr":"172.21.15.204:6835","nonce":1793346439}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5082,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/524822971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":524822971},{"type":"v1","addr":"172.21.15.204:6833","nonce":524822971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5085,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/831847190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":831847190},{"type":"v1","addr":"172.21.15.143:6835","nonce":831847190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-16T15:01:19.290571+0000","modified":"2024-01-16T15:01:22.368828+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5094},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5094":{"gid":5094,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.143:6837/4104468169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4104468169},{"type":"v1","addr":"172.21.15.143:6837","nonce":4104468169}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-01-16T15:01:23.357 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 17 2024-01-16T15:01:23.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.379+0000 7fe966600640 1 -- 172.21.15.143:0/3864831184 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe93806b2f0 msgr2=0x7fe93806d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:23.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.379+0000 7fe966600640 1 --2- 172.21.15.143:0/3864831184 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe93806b2f0 0x7fe93806d7b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe948036000 tx=0x7fe94803c040 comp rx=0 tx=0).stop 2024-01-16T15:01:23.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.379+0000 7fe966600640 1 -- 172.21.15.143:0/3864831184 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9600d0070 msgr2=0x7fe9600b5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:23.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.379+0000 7fe966600640 1 --2- 172.21.15.143:0/3864831184 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9600d0070 0x7fe9600b5df0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe95000b410 tx=0x7fe95000b8d0 comp rx=0 tx=0).stop 2024-01-16T15:01:23.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.379+0000 7fe966600640 1 -- 172.21.15.143:0/3864831184 shutdown_connections 2024-01-16T15:01:23.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.379+0000 7fe966600640 1 --2- 172.21.15.143:0/3864831184 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe93806b2f0 0x7fe93806d7b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:23.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.379+0000 7fe966600640 1 --2- 172.21.15.143:0/3864831184 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe96015d4e0 0x7fe9600b76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:23.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.379+0000 7fe966600640 1 --2- 172.21.15.143:0/3864831184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9600d38e0 0x7fe9600c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:23.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.379+0000 7fe966600640 1 --2- 172.21.15.143:0/3864831184 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9600d0070 0x7fe9600b5df0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:23.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.379+0000 7fe966600640 1 -- 172.21.15.143:0/3864831184 >> 172.21.15.143:0/3864831184 conn(0x7fe96009c510 msgr2=0x7fe96006b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:23.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.379+0000 7fe966600640 1 -- 172.21.15.143:0/3864831184 shutdown_connections 2024-01-16T15:01:23.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.379+0000 7fe966600640 1 -- 172.21.15.143:0/3864831184 wait complete. 2024-01-16T15:01:23.376 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T15:01:23.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.596+0000 7f349be00640 1 Processor -- start 2024-01-16T15:01:23.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.596+0000 7f349be00640 1 -- start start 2024-01-16T15:01:23.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.597+0000 7f349be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f349c06d5c0 0x7f349c071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:23.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.597+0000 7f349be00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f349c15a970 0x7f349c163430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:23.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.597+0000 7f349be00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f349c0ce870 con 0x7f349c159e80 2024-01-16T15:01:23.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.597+0000 7f349be00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f349c0cdc60 con 0x7f349c15a970 2024-01-16T15:01:23.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.597+0000 7f349be00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f349c0cdf40 con 0x7f349c06d5c0 2024-01-16T15:01:23.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.598+0000 7f3491e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f349c06d5c0 0x7f349c071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:23.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.598+0000 7f349b400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f349c15a970 0x7f349c163430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:23.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.598+0000 7f3491e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f349c06d5c0 0x7f349c071610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:43358/0 (socket says 172.21.15.143:43358) 2024-01-16T15:01:23.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.598+0000 7f3491e00640 1 -- 172.21.15.143:0/1083197106 learned_addr learned my addr 172.21.15.143:0/1083197106 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:23.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.599+0000 7f3491e00640 1 -- 172.21.15.143:0/1083197106 >> v2:172.21.15.204:3300/0 conn(0x7f349c15a970 msgr2=0x7f349c163430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:23.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.599+0000 7f3491e00640 1 --2- 172.21.15.143:0/1083197106 >> v2:172.21.15.204:3300/0 conn(0x7f349c15a970 0x7f349c163430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:23.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.599+0000 7f3491e00640 1 -- 172.21.15.143:0/1083197106 >> v1:172.21.15.204:6789/0 conn(0x7f349c159e80 legacy=0x7f349c15a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:23.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.599+0000 7f3491e00640 1 -- 172.21.15.143:0/1083197106 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f349c0ce220 con 0x7f349c06d5c0 2024-01-16T15:01:23.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.600+0000 7f3491e00640 1 --2- 172.21.15.143:0/1083197106 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f349c06d5c0 0x7f349c071610 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3484009870 tx=0x7f348402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=289f60e0cca58f5c server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:23.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.601+0000 7f349a000640 1 -- 172.21.15.143:0/1083197106 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3484031640 con 0x7f349c06d5c0 2024-01-16T15:01:23.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.601+0000 7f349a000640 1 -- 172.21.15.143:0/1083197106 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34840317a0 con 0x7f349c06d5c0 2024-01-16T15:01:23.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.601+0000 7f349a000640 1 -- 172.21.15.143:0/1083197106 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3484031b30 con 0x7f349c06d5c0 2024-01-16T15:01:23.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.601+0000 7f349be00640 1 -- 172.21.15.143:0/1083197106 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f349c06d5c0 msgr2=0x7f349c071610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:23.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.601+0000 7f349be00640 1 --2- 172.21.15.143:0/1083197106 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f349c06d5c0 0x7f349c071610 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3484009870 tx=0x7f348402b8f0 comp rx=0 tx=0).stop 2024-01-16T15:01:23.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.601+0000 7f349be00640 1 -- 172.21.15.143:0/1083197106 shutdown_connections 2024-01-16T15:01:23.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.601+0000 7f349be00640 1 --2- 172.21.15.143:0/1083197106 >> v2:172.21.15.204:3300/0 conn(0x7f349c15a970 0x7f349c163430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:23.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.601+0000 7f349be00640 1 --2- 172.21.15.143:0/1083197106 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f349c06d5c0 0x7f349c071610 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:23.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.601+0000 7f349be00640 1 -- 172.21.15.143:0/1083197106 >> 172.21.15.143:0/1083197106 conn(0x7f349c0cb830 msgr2=0x7f349c0cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:23.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.602+0000 7f349be00640 1 -- 172.21.15.143:0/1083197106 shutdown_connections 2024-01-16T15:01:23.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.602+0000 7f349be00640 1 -- 172.21.15.143:0/1083197106 wait complete. 2024-01-16T15:01:23.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.604+0000 7f349be00640 1 Processor -- start 2024-01-16T15:01:23.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.604+0000 7f349be00640 1 -- start start 2024-01-16T15:01:23.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.605+0000 7f349be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f349c06d5c0 0x7f349c143ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:23.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.605+0000 7f349be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f349c159e80 0x7f349c142740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:23.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.605+0000 7f349be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f349c15a970 0x7f349c14b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:23.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.605+0000 7f349be00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349c14ba90 con 0x7f349c06d5c0 2024-01-16T15:01:23.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.605+0000 7f349be00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349c143a50 con 0x7f349c159e80 2024-01-16T15:01:23.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.605+0000 7f349be00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f349c143120 con 0x7f349c15a970 2024-01-16T15:01:23.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.605+0000 7f349aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f349c06d5c0 0x7f349c143ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:23.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.605+0000 7f349aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f349c06d5c0 0x7f349c143ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59190/0 (socket says 172.21.15.143:59190) 2024-01-16T15:01:23.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.605+0000 7f349aa00640 1 -- 172.21.15.143:0/4226675202 learned_addr learned my addr 172.21.15.143:0/4226675202 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:23.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.605+0000 7f3491e00640 1 --2- 172.21.15.143:0/4226675202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f349c159e80 0x7f349c142740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:23.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.606+0000 7f349b400640 1 --2- 172.21.15.143:0/4226675202 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f349c15a970 0x7f349c14b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:23.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.606+0000 7f349aa00640 1 -- 172.21.15.143:0/4226675202 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f349c15a970 msgr2=0x7f349c14b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:23.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.606+0000 7f349aa00640 1 --2- 172.21.15.143:0/4226675202 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f349c15a970 0x7f349c14b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:23.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.606+0000 7f349aa00640 1 -- 172.21.15.143:0/4226675202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f349c159e80 msgr2=0x7f349c142740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:23.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.606+0000 7f349aa00640 1 --2- 172.21.15.143:0/4226675202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f349c159e80 0x7f349c142740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:23.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.606+0000 7f349aa00640 1 -- 172.21.15.143:0/4226675202 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34840094d0 con 0x7f349c06d5c0 2024-01-16T15:01:23.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.606+0000 7f349b400640 1 --2- 172.21.15.143:0/4226675202 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f349c15a970 0x7f349c14b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:01:23.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.606+0000 7f349aa00640 1 --2- 172.21.15.143:0/4226675202 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f349c06d5c0 0x7f349c143ee0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f348c00c400 tx=0x7f348c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:23.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.606+0000 7f3493400640 1 -- 172.21.15.143:0/4226675202 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348c0052d0 con 0x7f349c06d5c0 2024-01-16T15:01:23.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.606+0000 7f3493400640 1 -- 172.21.15.143:0/4226675202 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f348c005430 con 0x7f349c06d5c0 2024-01-16T15:01:23.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.606+0000 7f3493400640 1 -- 172.21.15.143:0/4226675202 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348c003830 con 0x7f349c06d5c0 2024-01-16T15:01:23.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.606+0000 7f349be00640 1 -- 172.21.15.143:0/4226675202 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f349c143400 con 0x7f349c06d5c0 2024-01-16T15:01:23.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.606+0000 7f349be00640 1 -- 172.21.15.143:0/4226675202 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f349c142c80 con 0x7f349c06d5c0 2024-01-16T15:01:23.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.608+0000 7f349be00640 1 -- 172.21.15.143:0/4226675202 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f349c04e840 con 0x7f349c06d5c0 2024-01-16T15:01:23.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.609+0000 7f3493400640 1 -- 172.21.15.143:0/4226675202 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f348c003990 con 0x7f349c06d5c0 2024-01-16T15:01:23.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.609+0000 7f3493400640 1 --2- 172.21.15.143:0/4226675202 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f347406b2c0 0x7f347406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:23.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.610+0000 7f3493400640 1 -- 172.21.15.143:0/4226675202 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f348c0890a0 con 0x7f349c06d5c0 2024-01-16T15:01:23.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.610+0000 7f3491e00640 1 --2- 172.21.15.143:0/4226675202 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f347406b2c0 0x7f347406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:23.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.610+0000 7f3491e00640 1 --2- 172.21.15.143:0/4226675202 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f347406b2c0 0x7f347406d780 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3484036000 tx=0x7f348403c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:23.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.614+0000 7f3493400640 1 -- 172.21.15.143:0/4226675202 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f348c053fa0 con 0x7f349c06d5c0 2024-01-16T15:01:23.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.806+0000 7f349be00640 1 -- 172.21.15.143:0/4226675202 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f349c0b0500 con 0x7f349c06d5c0 2024-01-16T15:01:23.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.808+0000 7f3493400640 1 -- 172.21.15.143:0/4226675202 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16629 (secure 0 0 0) 0x7f348c0575c0 con 0x7f349c06d5c0 2024-01-16T15:01:23.789 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:01:23.789 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":34,"fsid":"c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0","created":"2024-01-16T14:59:38.532601+0000","modified":"2024-01-16T15:01:20.881911+0000","last_up_change":"2024-01-16T14:59:50.983699+0000","last_in_change":"2024-01-16T14:59:42.056297+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:59:51.727720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-01-16T15:01:17.024625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-01-16T15:01:18.326532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f4c8891d-ab29-472a-907d-8446f20f7012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6801","nonce":1569293626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6805","nonce":1569293626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6814","nonce":1569293626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6809","nonce":1569293626}]},"public_addr":"172.21.15.143:6801/1569293626","cluster_addr":"172.21.15.143:6805/1569293626","heartbeat_back_addr":"172.21.15.143:6814/1569293626","heartbeat_front_addr":"172.21.15.143:6809/1569293626","state":["exists","up"]},{"osd":1,"uuid":"c4a2788d-8b89-4992-82b2-7f4b9debd509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6804","nonce":1276841168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6808","nonce":1276841168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6813","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6815","nonce":1276841168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6811","nonce":1276841168}]},"public_addr":"172.21.15.143:6804/1276841168","cluster_addr":"172.21.15.143:6808/1276841168","heartbeat_back_addr":"172.21.15.143:6815/1276841168","heartbeat_front_addr":"172.21.15.143:6811/1276841168","state":["exists","up"]},{"osd":2,"uuid":"64645c1d-31b2-4794-b0f2-7ccf8df30393","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6817","nonce":1217670170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6819","nonce":1217670170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6823","nonce":1217670170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6821","nonce":1217670170}]},"public_addr":"172.21.15.143:6817/1217670170","cluster_addr":"172.21.15.143:6819/1217670170","heartbeat_back_addr":"172.21.15.143:6823/1217670170","heartbeat_front_addr":"172.21.15.143:6821/1217670170","state":["exists","up"]},{"osd":3,"uuid":"73674129-a84e-4d31-8247-7d0179ae6df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6825","nonce":3414188012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6827","nonce":3414188012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6831","nonce":3414188012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6829","nonce":3414188012}]},"public_addr":"172.21.15.143:6825/3414188012","cluster_addr":"172.21.15.143:6827/3414188012","heartbeat_back_addr":"172.21.15.143:6831/3414188012","heartbeat_front_addr":"172.21.15.143:6829/3414188012","state":["exists","up"]},{"osd":4,"uuid":"cea28c9b-2f1c-4087-b415-9960789131d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6811","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6812","nonce":1070255716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6816","nonce":1070255716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6823","nonce":1070255716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6820","nonce":1070255716}]},"public_addr":"172.21.15.204:6812/1070255716","cluster_addr":"172.21.15.204:6816/1070255716","heartbeat_back_addr":"172.21.15.204:6823/1070255716","heartbeat_front_addr":"172.21.15.204:6820/1070255716","state":["exists","up"]},{"osd":5,"uuid":"33b63560-5574-4e78-983f-b398d67a2cb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6809","nonce":3605417837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6813","nonce":3605417837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6819","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6821","nonce":3605417837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6817","nonce":3605417837}]},"public_addr":"172.21.15.204:6809/3605417837","cluster_addr":"172.21.15.204:6813/3605417837","heartbeat_back_addr":"172.21.15.204:6821/3605417837","heartbeat_front_addr":"172.21.15.204:6817/3605417837","state":["exists","up"]},{"osd":6,"uuid":"df397427-8940-4eee-9c79-eece669d735d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6801","nonce":1085000534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6803","nonce":1085000534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6808","nonce":1085000534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6805","nonce":1085000534}]},"public_addr":"172.21.15.204:6801/1085000534","cluster_addr":"172.21.15.204:6803/1085000534","heartbeat_back_addr":"172.21.15.204:6808/1085000534","heartbeat_front_addr":"172.21.15.204:6805/1085000534","state":["exists","up"]},{"osd":7,"uuid":"a2e5a15c-8312-405a-ac02-f7e1fc9df2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6825","nonce":1337220609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6827","nonce":1337220609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6831","nonce":1337220609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6829","nonce":1337220609}]},"public_addr":"172.21.15.204:6825/1337220609","cluster_addr":"172.21.15.204:6827/1337220609","heartbeat_back_addr":"172.21.15.204:6831/1337220609","heartbeat_front_addr":"172.21.15.204:6829/1337220609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:49.984507+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.395306+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.632998+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.373627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.342920+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.438366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.331926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.406657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T15:01:23.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.811+0000 7f349be00640 1 -- 172.21.15.143:0/4226675202 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f347406b2c0 msgr2=0x7f347406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:23.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.811+0000 7f349be00640 1 --2- 172.21.15.143:0/4226675202 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f347406b2c0 0x7f347406d780 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3484036000 tx=0x7f348403c040 comp rx=0 tx=0).stop 2024-01-16T15:01:23.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.811+0000 7f349be00640 1 -- 172.21.15.143:0/4226675202 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f349c06d5c0 msgr2=0x7f349c143ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:23.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.811+0000 7f349be00640 1 --2- 172.21.15.143:0/4226675202 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f349c06d5c0 0x7f349c143ee0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f348c00c400 tx=0x7f348c00c8c0 comp rx=0 tx=0).stop 2024-01-16T15:01:23.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.811+0000 7f349be00640 1 -- 172.21.15.143:0/4226675202 shutdown_connections 2024-01-16T15:01:23.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.811+0000 7f349be00640 1 --2- 172.21.15.143:0/4226675202 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f347406b2c0 0x7f347406d780 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:23.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.811+0000 7f349be00640 1 --2- 172.21.15.143:0/4226675202 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f349c15a970 0x7f349c14b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:23.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.811+0000 7f349be00640 1 --2- 172.21.15.143:0/4226675202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f349c159e80 0x7f349c142740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:23.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.811+0000 7f349be00640 1 --2- 172.21.15.143:0/4226675202 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f349c06d5c0 0x7f349c143ee0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:23.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.811+0000 7f349be00640 1 -- 172.21.15.143:0/4226675202 >> 172.21.15.143:0/4226675202 conn(0x7f349c0cb830 msgr2=0x7f349c151620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:23.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.811+0000 7f349be00640 1 -- 172.21.15.143:0/4226675202 shutdown_connections 2024-01-16T15:01:23.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:23.811+0000 7f349be00640 1 -- 172.21.15.143:0/4226675202 wait complete. 2024-01-16T15:01:23.810 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-16T15:01:23.810 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-16T15:01:24.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.028+0000 7fedeb200640 1 Processor -- start 2024-01-16T15:01:24.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.028+0000 7fedeb200640 1 -- start start 2024-01-16T15:01:24.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.030+0000 7fedeb200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fede415c7f0 0x7fede406db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:24.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.030+0000 7fedeb200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fede40cf450 0x7fede40cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:24.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.030+0000 7fedeb200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fede40cece0 con 0x7fede40cdfb0 2024-01-16T15:01:24.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.030+0000 7fedeb200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fede40cfe40 con 0x7fede415c7f0 2024-01-16T15:01:24.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.030+0000 7fedeb200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fede40ceaf0 con 0x7fede40cf450 2024-01-16T15:01:24.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.030+0000 7fede3e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fede415c7f0 0x7fede406db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:24.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.030+0000 7fede9200640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fede40cdfb0 0x7fede40ce3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57802/0 (socket says 172.21.15.143:57802) 2024-01-16T15:01:24.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.030+0000 7fede9200640 1 -- 172.21.15.143:0/1023098215 learned_addr learned my addr 172.21.15.143:0/1023098215 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:01:24.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.031+0000 7feddb200640 1 --2- 172.21.15.143:0/1023098215 >> v2:172.21.15.204:3300/0 conn(0x7fede40cf450 0x7fede40cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:24.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.031+0000 7fede3400640 1 -- 172.21.15.143:0/1023098215 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 630451265 0 0) 0x7fede40cece0 con 0x7fede40cdfb0 2024-01-16T15:01:24.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.031+0000 7fede3e00640 1 -- 172.21.15.143:0/1023098215 >> v2:172.21.15.204:3300/0 conn(0x7fede40cf450 msgr2=0x7fede40cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:24.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.031+0000 7fede3e00640 1 --2- 172.21.15.143:0/1023098215 >> v2:172.21.15.204:3300/0 conn(0x7fede40cf450 0x7fede40cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:24.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.031+0000 7fede3e00640 1 -- 172.21.15.143:0/1023098215 >> v1:172.21.15.143:6789/0 conn(0x7fede40cdfb0 legacy=0x7fede40ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:24.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.031+0000 7fede3e00640 1 -- 172.21.15.143:0/1023098215 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede406d040 con 0x7fede415c7f0 2024-01-16T15:01:24.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.032+0000 7fede3e00640 1 --2- 172.21.15.143:0/1023098215 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fede415c7f0 0x7fede406db20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fede4070410 tx=0x7fedd402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1880ee3b8f2daa7c server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:24.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.032+0000 7fede3400640 1 -- 172.21.15.143:0/1023098215 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd4031830 con 0x7fede415c7f0 2024-01-16T15:01:24.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.032+0000 7fede3400640 1 -- 172.21.15.143:0/1023098215 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedd4031990 con 0x7fede415c7f0 2024-01-16T15:01:24.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.032+0000 7fede3400640 1 -- 172.21.15.143:0/1023098215 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd4031d20 con 0x7fede415c7f0 2024-01-16T15:01:24.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.033+0000 7fedeb200640 1 -- 172.21.15.143:0/1023098215 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fede415c7f0 msgr2=0x7fede406db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:24.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.033+0000 7fedeb200640 1 --2- 172.21.15.143:0/1023098215 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fede415c7f0 0x7fede406db20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fede4070410 tx=0x7fedd402b8d0 comp rx=0 tx=0).stop 2024-01-16T15:01:24.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.033+0000 7fedeb200640 1 -- 172.21.15.143:0/1023098215 shutdown_connections 2024-01-16T15:01:24.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.033+0000 7fedeb200640 1 --2- 172.21.15.143:0/1023098215 >> v2:172.21.15.204:3300/0 conn(0x7fede40cf450 0x7fede40cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:24.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.033+0000 7fedeb200640 1 --2- 172.21.15.143:0/1023098215 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fede415c7f0 0x7fede406db20 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:24.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.033+0000 7fedeb200640 1 -- 172.21.15.143:0/1023098215 >> 172.21.15.143:0/1023098215 conn(0x7fede406a920 msgr2=0x7fede406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:24.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.033+0000 7fedeb200640 1 -- 172.21.15.143:0/1023098215 shutdown_connections 2024-01-16T15:01:24.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.033+0000 7fedeb200640 1 -- 172.21.15.143:0/1023098215 wait complete. 2024-01-16T15:01:24.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.035+0000 7fedeb200640 1 Processor -- start 2024-01-16T15:01:24.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.035+0000 7fedeb200640 1 -- start start 2024-01-16T15:01:24.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.036+0000 7fedeb200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fede40cdfb0 0x7fede414bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:24.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.036+0000 7fedeb200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fede40cf450 0x7fede40abfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:24.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.036+0000 7fedeb200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fede415c7f0 0x7fede40b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:24.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.036+0000 7fedeb200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede40ad240 con 0x7fede40cdfb0 2024-01-16T15:01:24.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.036+0000 7fedeb200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede41535b0 con 0x7fede40cf450 2024-01-16T15:01:24.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.036+0000 7fedeb200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fede414c240 con 0x7fede415c7f0 2024-01-16T15:01:24.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.036+0000 7fede3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fede40cdfb0 0x7fede414bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:24.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.036+0000 7fede3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fede40cdfb0 0x7fede414bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59222/0 (socket says 172.21.15.143:59222) 2024-01-16T15:01:24.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.036+0000 7fede3e00640 1 -- 172.21.15.143:0/4018001461 learned_addr learned my addr 172.21.15.143:0/4018001461 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:24.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.037+0000 7feddb200640 1 --2- 172.21.15.143:0/4018001461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fede40cf450 0x7fede40abfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:24.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.037+0000 7fede9200640 1 --2- 172.21.15.143:0/4018001461 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fede415c7f0 0x7fede40b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:24.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.037+0000 7fede3e00640 1 -- 172.21.15.143:0/4018001461 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fede415c7f0 msgr2=0x7fede40b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:24.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.037+0000 7fede3e00640 1 --2- 172.21.15.143:0/4018001461 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fede415c7f0 0x7fede40b2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:24.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.037+0000 7fede3e00640 1 -- 172.21.15.143:0/4018001461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fede40cf450 msgr2=0x7fede40abfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:24.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.037+0000 7fede3e00640 1 --2- 172.21.15.143:0/4018001461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fede40cf450 0x7fede40abfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:24.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.037+0000 7fede3e00640 1 -- 172.21.15.143:0/4018001461 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd4009400 con 0x7fede40cdfb0 2024-01-16T15:01:24.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.037+0000 7fede9200640 1 --2- 172.21.15.143:0/4018001461 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fede415c7f0 0x7fede40b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:01:24.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.037+0000 7fede3e00640 1 --2- 172.21.15.143:0/4018001461 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fede40cdfb0 0x7fede414bd00 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fedd4000c00 tx=0x7fedd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:24.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.037+0000 7fede0c00640 1 -- 172.21.15.143:0/4018001461 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd403aa60 con 0x7fede40cdfb0 2024-01-16T15:01:24.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.037+0000 7fede0c00640 1 -- 172.21.15.143:0/4018001461 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedd403abe0 con 0x7fede40cdfb0 2024-01-16T15:01:24.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.037+0000 7fede0c00640 1 -- 172.21.15.143:0/4018001461 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd40443f0 con 0x7fede40cdfb0 2024-01-16T15:01:24.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.038+0000 7fedeb200640 1 -- 172.21.15.143:0/4018001461 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fede40ac9a0 con 0x7fede40cdfb0 2024-01-16T15:01:24.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.038+0000 7fedeb200640 1 -- 172.21.15.143:0/4018001461 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fede40acf60 con 0x7fede40cdfb0 2024-01-16T15:01:24.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.039+0000 7fedeb200640 1 -- 172.21.15.143:0/4018001461 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fede4153310 con 0x7fede40cdfb0 2024-01-16T15:01:24.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.040+0000 7fede0c00640 1 -- 172.21.15.143:0/4018001461 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fedd4044550 con 0x7fede40cdfb0 2024-01-16T15:01:24.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.040+0000 7fede0c00640 1 --2- 172.21.15.143:0/4018001461 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fedbc06b2c0 0x7fedbc06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:24.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.041+0000 7fede0c00640 1 -- 172.21.15.143:0/4018001461 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fedd40abfe0 con 0x7fede40cdfb0 2024-01-16T15:01:24.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.041+0000 7feddb200640 1 --2- 172.21.15.143:0/4018001461 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fedbc06b2c0 0x7fedbc06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:24.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.041+0000 7feddb200640 1 --2- 172.21.15.143:0/4018001461 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fedbc06b2c0 0x7fedbc06d780 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fede40b2960 tx=0x7fedcc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:24.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.045+0000 7fede0c00640 1 -- 172.21.15.143:0/4018001461 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fede4153310 con 0x7fede40cdfb0 2024-01-16T15:01:24.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.230+0000 7fedeb200640 1 -- 172.21.15.143:0/4018001461 --> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fede406e060 con 0x7fedbc06b2c0 2024-01-16T15:01:24.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.259+0000 7fede0c00640 1 -- 172.21.15.143:0/4018001461 <== mgr.4113 v2:172.21.15.143:6832/3153492115 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287070 (secure 0 0 0) 0x7fedcc009d60 con 0x7fedbc06b2c0 2024-01-16T15:01:24.243 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:01:24.245 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-16T15:01:24.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.266+0000 7fedeb200640 1 -- 172.21.15.143:0/4018001461 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fedbc06b2c0 msgr2=0x7fedbc06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:24.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.266+0000 7fedeb200640 1 --2- 172.21.15.143:0/4018001461 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fedbc06b2c0 0x7fedbc06d780 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fede40b2960 tx=0x7fedcc008040 comp rx=0 tx=0).stop 2024-01-16T15:01:24.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.266+0000 7fedeb200640 1 -- 172.21.15.143:0/4018001461 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fede40cdfb0 msgr2=0x7fede414bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:24.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.266+0000 7fedeb200640 1 --2- 172.21.15.143:0/4018001461 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fede40cdfb0 0x7fede414bd00 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fedd4000c00 tx=0x7fedd40046c0 comp rx=0 tx=0).stop 2024-01-16T15:01:24.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.266+0000 7fedeb200640 1 -- 172.21.15.143:0/4018001461 shutdown_connections 2024-01-16T15:01:24.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.266+0000 7fedeb200640 1 --2- 172.21.15.143:0/4018001461 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fedbc06b2c0 0x7fedbc06d780 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:24.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.266+0000 7fedeb200640 1 --2- 172.21.15.143:0/4018001461 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fede415c7f0 0x7fede40b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:24.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.266+0000 7fedeb200640 1 --2- 172.21.15.143:0/4018001461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fede40cf450 0x7fede40abfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:24.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.266+0000 7fedeb200640 1 --2- 172.21.15.143:0/4018001461 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fede40cdfb0 0x7fede414bd00 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:24.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.266+0000 7fedeb200640 1 -- 172.21.15.143:0/4018001461 >> 172.21.15.143:0/4018001461 conn(0x7fede406a920 msgr2=0x7fede40ce3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:24.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.266+0000 7fedeb200640 1 -- 172.21.15.143:0/4018001461 shutdown_connections 2024-01-16T15:01:24.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.266+0000 7fedeb200640 1 -- 172.21.15.143:0/4018001461 wait complete. 2024-01-16T15:01:24.264 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2024-01-16T15:01:23.677277+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220380,"kb_used_data":4180,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754340,"statfs":{"total":773094113280,"available":772868444160,"internally_reserved":0,"allocated":4280320,"data_stored":2374529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.846099"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.445114+0000","last_change":"2024-01-16T15:01:19.896749+0000","last_active":"2024-01-16T15:01:21.445114+0000","last_peered":"2024-01-16T15:01:21.445114+0000","last_clean":"2024-01-16T15:01:21.445114+0000","last_became_active":"2024-01-16T15:01:19.896443+0000","last_became_peered":"2024-01-16T15:01:19.896443+0000","last_unstale":"2024-01-16T15:01:21.445114+0000","last_undegraded":"2024-01-16T15:01:21.445114+0000","last_fullsized":"2024-01-16T15:01:21.445114+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:36:47.158317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.898297+0000","last_change":"2024-01-16T15:01:19.905287+0000","last_active":"2024-01-16T15:01:20.898297+0000","last_peered":"2024-01-16T15:01:20.898297+0000","last_clean":"2024-01-16T15:01:20.898297+0000","last_became_active":"2024-01-16T15:01:19.904981+0000","last_became_peered":"2024-01-16T15:01:19.904981+0000","last_unstale":"2024-01-16T15:01:20.898297+0000","last_undegraded":"2024-01-16T15:01:20.898297+0000","last_fullsized":"2024-01-16T15:01:20.898297+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:49:04.111811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.901813+0000","last_change":"2024-01-16T15:01:19.905331+0000","last_active":"2024-01-16T15:01:20.901813+0000","last_peered":"2024-01-16T15:01:20.901813+0000","last_clean":"2024-01-16T15:01:20.901813+0000","last_became_active":"2024-01-16T15:01:19.904652+0000","last_became_peered":"2024-01-16T15:01:19.904652+0000","last_unstale":"2024-01-16T15:01:20.901813+0000","last_undegraded":"2024-01-16T15:01:20.901813+0000","last_fullsized":"2024-01-16T15:01:20.901813+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:31:08.299586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.444269+0000","last_change":"2024-01-16T15:01:19.897610+0000","last_active":"2024-01-16T15:01:21.444269+0000","last_peered":"2024-01-16T15:01:21.444269+0000","last_clean":"2024-01-16T15:01:21.444269+0000","last_became_active":"2024-01-16T15:01:19.897301+0000","last_became_peered":"2024-01-16T15:01:19.897301+0000","last_unstale":"2024-01-16T15:01:21.444269+0000","last_undegraded":"2024-01-16T15:01:21.444269+0000","last_fullsized":"2024-01-16T15:01:21.444269+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:30:42.530766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.349263+0000","last_change":"2024-01-16T15:01:19.903993+0000","last_active":"2024-01-16T15:01:21.349263+0000","last_peered":"2024-01-16T15:01:21.349263+0000","last_clean":"2024-01-16T15:01:21.349263+0000","last_became_active":"2024-01-16T15:01:19.902007+0000","last_became_peered":"2024-01-16T15:01:19.902007+0000","last_unstale":"2024-01-16T15:01:21.349263+0000","last_undegraded":"2024-01-16T15:01:21.349263+0000","last_fullsized":"2024-01-16T15:01:21.349263+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:37:11.890430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.897688+0000","last_change":"2024-01-16T15:01:19.890431+0000","last_active":"2024-01-16T15:01:20.897688+0000","last_peered":"2024-01-16T15:01:20.897688+0000","last_clean":"2024-01-16T15:01:20.897688+0000","last_became_active":"2024-01-16T15:01:19.890135+0000","last_became_peered":"2024-01-16T15:01:19.890135+0000","last_unstale":"2024-01-16T15:01:20.897688+0000","last_undegraded":"2024-01-16T15:01:20.897688+0000","last_fullsized":"2024-01-16T15:01:20.897688+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:01:00.718830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.903192+0000","last_change":"2024-01-16T15:01:19.895013+0000","last_active":"2024-01-16T15:01:20.903192+0000","last_peered":"2024-01-16T15:01:20.903192+0000","last_clean":"2024-01-16T15:01:20.903192+0000","last_became_active":"2024-01-16T15:01:19.894728+0000","last_became_peered":"2024-01-16T15:01:19.894728+0000","last_unstale":"2024-01-16T15:01:20.903192+0000","last_undegraded":"2024-01-16T15:01:20.903192+0000","last_fullsized":"2024-01-16T15:01:20.903192+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:12:08.205887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.902266+0000","last_change":"2024-01-16T15:01:19.897961+0000","last_active":"2024-01-16T15:01:20.902266+0000","last_peered":"2024-01-16T15:01:20.902266+0000","last_clean":"2024-01-16T15:01:20.902266+0000","last_became_active":"2024-01-16T15:01:19.894706+0000","last_became_peered":"2024-01-16T15:01:19.894706+0000","last_unstale":"2024-01-16T15:01:20.902266+0000","last_undegraded":"2024-01-16T15:01:20.902266+0000","last_fullsized":"2024-01-16T15:01:20.902266+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:08:18.347887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.446418+0000","last_change":"2024-01-16T15:01:19.903915+0000","last_active":"2024-01-16T15:01:21.446418+0000","last_peered":"2024-01-16T15:01:21.446418+0000","last_clean":"2024-01-16T15:01:21.446418+0000","last_became_active":"2024-01-16T15:01:19.903168+0000","last_became_peered":"2024-01-16T15:01:19.903168+0000","last_unstale":"2024-01-16T15:01:21.446418+0000","last_undegraded":"2024-01-16T15:01:21.446418+0000","last_fullsized":"2024-01-16T15:01:21.446418+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:08:18.347887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.444606+0000","last_change":"2024-01-16T15:01:19.894997+0000","last_active":"2024-01-16T15:01:21.444606+0000","last_peered":"2024-01-16T15:01:21.444606+0000","last_clean":"2024-01-16T15:01:21.444606+0000","last_became_active":"2024-01-16T15:01:19.894687+0000","last_became_peered":"2024-01-16T15:01:19.894687+0000","last_unstale":"2024-01-16T15:01:21.444606+0000","last_undegraded":"2024-01-16T15:01:21.444606+0000","last_fullsized":"2024-01-16T15:01:21.444606+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:41:02.260826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.904329+0000","last_change":"2024-01-16T15:01:19.902052+0000","last_active":"2024-01-16T15:01:20.904329+0000","last_peered":"2024-01-16T15:01:20.904329+0000","last_clean":"2024-01-16T15:01:20.904329+0000","last_became_active":"2024-01-16T15:01:19.901164+0000","last_became_peered":"2024-01-16T15:01:19.901164+0000","last_unstale":"2024-01-16T15:01:20.904329+0000","last_undegraded":"2024-01-16T15:01:20.904329+0000","last_fullsized":"2024-01-16T15:01:20.904329+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:49:04.111811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.901399+0000","last_change":"2024-01-16T15:01:19.905501+0000","last_active":"2024-01-16T15:01:20.901399+0000","last_peered":"2024-01-16T15:01:20.901399+0000","last_clean":"2024-01-16T15:01:20.901399+0000","last_became_active":"2024-01-16T15:01:19.905206+0000","last_became_peered":"2024-01-16T15:01:19.905206+0000","last_unstale":"2024-01-16T15:01:20.901399+0000","last_undegraded":"2024-01-16T15:01:20.901399+0000","last_fullsized":"2024-01-16T15:01:20.901399+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:08:18.347887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.349553+0000","last_change":"2024-01-16T15:01:19.904347+0000","last_active":"2024-01-16T15:01:21.349553+0000","last_peered":"2024-01-16T15:01:21.349553+0000","last_clean":"2024-01-16T15:01:21.349553+0000","last_became_active":"2024-01-16T15:01:19.902780+0000","last_became_peered":"2024-01-16T15:01:19.902780+0000","last_unstale":"2024-01-16T15:01:21.349553+0000","last_undegraded":"2024-01-16T15:01:21.349553+0000","last_fullsized":"2024-01-16T15:01:21.349553+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:41:22.619386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.897827+0000","last_change":"2024-01-16T15:01:19.894720+0000","last_active":"2024-01-16T15:01:20.897827+0000","last_peered":"2024-01-16T15:01:20.897827+0000","last_clean":"2024-01-16T15:01:20.897827+0000","last_became_active":"2024-01-16T15:01:19.894462+0000","last_became_peered":"2024-01-16T15:01:19.894462+0000","last_unstale":"2024-01-16T15:01:20.897827+0000","last_undegraded":"2024-01-16T15:01:20.897827+0000","last_fullsized":"2024-01-16T15:01:20.897827+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:52:44.965933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.900901+0000","last_change":"2024-01-16T15:01:19.894716+0000","last_active":"2024-01-16T15:01:20.900901+0000","last_peered":"2024-01-16T15:01:20.900901+0000","last_clean":"2024-01-16T15:01:20.900901+0000","last_became_active":"2024-01-16T15:01:19.894466+0000","last_became_peered":"2024-01-16T15:01:19.894466+0000","last_unstale":"2024-01-16T15:01:20.900901+0000","last_undegraded":"2024-01-16T15:01:20.900901+0000","last_fullsized":"2024-01-16T15:01:20.900901+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:41:02.260826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.902967+0000","last_change":"2024-01-16T15:01:19.888176+0000","last_active":"2024-01-16T15:01:20.902967+0000","last_peered":"2024-01-16T15:01:20.902967+0000","last_clean":"2024-01-16T15:01:20.902967+0000","last_became_active":"2024-01-16T15:01:19.886502+0000","last_became_peered":"2024-01-16T15:01:19.886502+0000","last_unstale":"2024-01-16T15:01:20.902967+0000","last_undegraded":"2024-01-16T15:01:20.902967+0000","last_fullsized":"2024-01-16T15:01:20.902967+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:01:00.718830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.903736+0000","last_change":"2024-01-16T15:01:19.898338+0000","last_active":"2024-01-16T15:01:20.903736+0000","last_peered":"2024-01-16T15:01:20.903736+0000","last_clean":"2024-01-16T15:01:20.903736+0000","last_became_active":"2024-01-16T15:01:19.898050+0000","last_became_peered":"2024-01-16T15:01:19.898050+0000","last_unstale":"2024-01-16T15:01:20.903736+0000","last_undegraded":"2024-01-16T15:01:20.903736+0000","last_fullsized":"2024-01-16T15:01:20.903736+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:02:00.425436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.898245+0000","last_change":"2024-01-16T15:01:19.915366+0000","last_active":"2024-01-16T15:01:20.898245+0000","last_peered":"2024-01-16T15:01:20.898245+0000","last_clean":"2024-01-16T15:01:20.898245+0000","last_became_active":"2024-01-16T15:01:19.913042+0000","last_became_peered":"2024-01-16T15:01:19.913042+0000","last_unstale":"2024-01-16T15:01:20.898245+0000","last_undegraded":"2024-01-16T15:01:20.898245+0000","last_fullsized":"2024-01-16T15:01:20.898245+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:25:30.380602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.898042+0000","last_change":"2024-01-16T15:01:19.904308+0000","last_active":"2024-01-16T15:01:20.898042+0000","last_peered":"2024-01-16T15:01:20.898042+0000","last_clean":"2024-01-16T15:01:20.898042+0000","last_became_active":"2024-01-16T15:01:19.904053+0000","last_became_peered":"2024-01-16T15:01:19.904053+0000","last_unstale":"2024-01-16T15:01:20.898042+0000","last_undegraded":"2024-01-16T15:01:20.898042+0000","last_fullsized":"2024-01-16T15:01:20.898042+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:47:30.255903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.349522+0000","last_change":"2024-01-16T15:01:19.902503+0000","last_active":"2024-01-16T15:01:21.349522+0000","last_peered":"2024-01-16T15:01:21.349522+0000","last_clean":"2024-01-16T15:01:21.349522+0000","last_became_active":"2024-01-16T15:01:19.901569+0000","last_became_peered":"2024-01-16T15:01:19.901569+0000","last_unstale":"2024-01-16T15:01:21.349522+0000","last_undegraded":"2024-01-16T15:01:21.349522+0000","last_fullsized":"2024-01-16T15:01:21.349522+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:03:57.467477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.444750+0000","last_change":"2024-01-16T15:01:19.898103+0000","last_active":"2024-01-16T15:01:21.444750+0000","last_peered":"2024-01-16T15:01:21.444750+0000","last_clean":"2024-01-16T15:01:21.444750+0000","last_became_active":"2024-01-16T15:01:19.897858+0000","last_became_peered":"2024-01-16T15:01:19.897858+0000","last_unstale":"2024-01-16T15:01:21.444750+0000","last_undegraded":"2024-01-16T15:01:21.444750+0000","last_fullsized":"2024-01-16T15:01:21.444750+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:30:42.530766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.444615+0000","last_change":"2024-01-16T15:01:19.894844+0000","last_active":"2024-01-16T15:01:21.444615+0000","last_peered":"2024-01-16T15:01:21.444615+0000","last_clean":"2024-01-16T15:01:21.444615+0000","last_became_active":"2024-01-16T15:01:19.893574+0000","last_became_peered":"2024-01-16T15:01:19.893574+0000","last_unstale":"2024-01-16T15:01:21.444615+0000","last_undegraded":"2024-01-16T15:01:21.444615+0000","last_fullsized":"2024-01-16T15:01:21.444615+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:46:28.675655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.442171+0000","last_change":"2024-01-16T15:01:19.877123+0000","last_active":"2024-01-16T15:01:21.442171+0000","last_peered":"2024-01-16T15:01:21.442171+0000","last_clean":"2024-01-16T15:01:21.442171+0000","last_became_active":"2024-01-16T15:01:19.876406+0000","last_became_peered":"2024-01-16T15:01:19.876406+0000","last_unstale":"2024-01-16T15:01:21.442171+0000","last_undegraded":"2024-01-16T15:01:21.442171+0000","last_fullsized":"2024-01-16T15:01:21.442171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:14:12.867834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.900932+0000","last_change":"2024-01-16T15:01:19.902197+0000","last_active":"2024-01-16T15:01:20.900932+0000","last_peered":"2024-01-16T15:01:20.900932+0000","last_clean":"2024-01-16T15:01:20.900932+0000","last_became_active":"2024-01-16T15:01:19.901878+0000","last_became_peered":"2024-01-16T15:01:19.901878+0000","last_unstale":"2024-01-16T15:01:20.900932+0000","last_undegraded":"2024-01-16T15:01:20.900932+0000","last_fullsized":"2024-01-16T15:01:20.900932+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:36:30.879279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.444785+0000","last_change":"2024-01-16T15:01:19.898875+0000","last_active":"2024-01-16T15:01:21.444785+0000","last_peered":"2024-01-16T15:01:21.444785+0000","last_clean":"2024-01-16T15:01:21.444785+0000","last_became_active":"2024-01-16T15:01:19.898575+0000","last_became_peered":"2024-01-16T15:01:19.898575+0000","last_unstale":"2024-01-16T15:01:21.444785+0000","last_undegraded":"2024-01-16T15:01:21.444785+0000","last_fullsized":"2024-01-16T15:01:21.444785+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:25:29.059223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.898387+0000","last_change":"2024-01-16T15:01:19.893928+0000","last_active":"2024-01-16T15:01:20.898387+0000","last_peered":"2024-01-16T15:01:20.898387+0000","last_clean":"2024-01-16T15:01:20.898387+0000","last_became_active":"2024-01-16T15:01:19.893678+0000","last_became_peered":"2024-01-16T15:01:19.893678+0000","last_unstale":"2024-01-16T15:01:20.898387+0000","last_undegraded":"2024-01-16T15:01:20.898387+0000","last_fullsized":"2024-01-16T15:01:20.898387+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:01:23.441562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.445123+0000","last_change":"2024-01-16T15:01:19.898480+0000","last_active":"2024-01-16T15:01:21.445123+0000","last_peered":"2024-01-16T15:01:21.445123+0000","last_clean":"2024-01-16T15:01:21.445123+0000","last_became_active":"2024-01-16T15:01:19.898161+0000","last_became_peered":"2024-01-16T15:01:19.898161+0000","last_unstale":"2024-01-16T15:01:21.445123+0000","last_undegraded":"2024-01-16T15:01:21.445123+0000","last_fullsized":"2024-01-16T15:01:21.445123+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:08:18.347887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.901900+0000","last_change":"2024-01-16T15:01:19.901348+0000","last_active":"2024-01-16T15:01:20.901900+0000","last_peered":"2024-01-16T15:01:20.901900+0000","last_clean":"2024-01-16T15:01:20.901900+0000","last_became_active":"2024-01-16T15:01:19.900965+0000","last_became_peered":"2024-01-16T15:01:19.900965+0000","last_unstale":"2024-01-16T15:01:20.901900+0000","last_undegraded":"2024-01-16T15:01:20.901900+0000","last_fullsized":"2024-01-16T15:01:20.901900+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:01:00.718830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.445039+0000","last_change":"2024-01-16T15:01:19.893658+0000","last_active":"2024-01-16T15:01:21.445039+0000","last_peered":"2024-01-16T15:01:21.445039+0000","last_clean":"2024-01-16T15:01:21.445039+0000","last_became_active":"2024-01-16T15:01:19.893342+0000","last_became_peered":"2024-01-16T15:01:19.893342+0000","last_unstale":"2024-01-16T15:01:21.445039+0000","last_undegraded":"2024-01-16T15:01:21.445039+0000","last_fullsized":"2024-01-16T15:01:21.445039+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:41:02.260826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.903326+0000","last_change":"2024-01-16T15:01:19.900634+0000","last_active":"2024-01-16T15:01:20.903326+0000","last_peered":"2024-01-16T15:01:20.903326+0000","last_clean":"2024-01-16T15:01:20.903326+0000","last_became_active":"2024-01-16T15:01:19.900337+0000","last_became_peered":"2024-01-16T15:01:19.900337+0000","last_unstale":"2024-01-16T15:01:20.903326+0000","last_undegraded":"2024-01-16T15:01:20.903326+0000","last_fullsized":"2024-01-16T15:01:20.903326+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:47:30.255903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.443980+0000","last_change":"2024-01-16T15:01:19.897073+0000","last_active":"2024-01-16T15:01:21.443980+0000","last_peered":"2024-01-16T15:01:21.443980+0000","last_clean":"2024-01-16T15:01:21.443980+0000","last_became_active":"2024-01-16T15:01:19.896788+0000","last_became_peered":"2024-01-16T15:01:19.896788+0000","last_unstale":"2024-01-16T15:01:21.443980+0000","last_undegraded":"2024-01-16T15:01:21.443980+0000","last_fullsized":"2024-01-16T15:01:21.443980+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:22:03.634466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.902293+0000","last_change":"2024-01-16T15:01:19.890441+0000","last_active":"2024-01-16T15:01:20.902293+0000","last_peered":"2024-01-16T15:01:20.902293+0000","last_clean":"2024-01-16T15:01:20.902293+0000","last_became_active":"2024-01-16T15:01:19.890076+0000","last_became_peered":"2024-01-16T15:01:19.890076+0000","last_unstale":"2024-01-16T15:01:20.902293+0000","last_undegraded":"2024-01-16T15:01:20.902293+0000","last_fullsized":"2024-01-16T15:01:20.902293+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:15:15.239596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.442964+0000","last_change":"2024-01-16T15:01:19.889683+0000","last_active":"2024-01-16T15:01:21.442964+0000","last_peered":"2024-01-16T15:01:21.442964+0000","last_clean":"2024-01-16T15:01:21.442964+0000","last_became_active":"2024-01-16T15:01:19.881670+0000","last_became_peered":"2024-01-16T15:01:19.881670+0000","last_unstale":"2024-01-16T15:01:21.442964+0000","last_undegraded":"2024-01-16T15:01:21.442964+0000","last_fullsized":"2024-01-16T15:01:21.442964+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:46:28.675655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.445661+0000","last_change":"2024-01-16T15:01:18.877659+0000","last_active":"2024-01-16T15:01:21.445661+0000","last_peered":"2024-01-16T15:01:21.445661+0000","last_clean":"2024-01-16T15:01:21.445661+0000","last_became_active":"2024-01-16T15:01:18.877324+0000","last_became_peered":"2024-01-16T15:01:18.877324+0000","last_unstale":"2024-01-16T15:01:21.445661+0000","last_undegraded":"2024-01-16T15:01:21.445661+0000","last_fullsized":"2024-01-16T15:01:21.445661+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:11:58.134390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.445482+0000","last_change":"2024-01-16T15:01:19.892754+0000","last_active":"2024-01-16T15:01:21.445482+0000","last_peered":"2024-01-16T15:01:21.445482+0000","last_clean":"2024-01-16T15:01:21.445482+0000","last_became_active":"2024-01-16T15:01:19.892461+0000","last_became_peered":"2024-01-16T15:01:19.892461+0000","last_unstale":"2024-01-16T15:01:21.445482+0000","last_undegraded":"2024-01-16T15:01:21.445482+0000","last_fullsized":"2024-01-16T15:01:21.445482+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:35:31.837292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.442834+0000","last_change":"2024-01-16T15:01:18.874834+0000","last_active":"2024-01-16T15:01:21.442834+0000","last_peered":"2024-01-16T15:01:21.442834+0000","last_clean":"2024-01-16T15:01:21.442834+0000","last_became_active":"2024-01-16T15:01:18.874602+0000","last_became_peered":"2024-01-16T15:01:18.874602+0000","last_unstale":"2024-01-16T15:01:21.442834+0000","last_undegraded":"2024-01-16T15:01:21.442834+0000","last_fullsized":"2024-01-16T15:01:21.442834+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:11:58.134390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.445916+0000","last_change":"2024-01-16T15:01:19.895059+0000","last_active":"2024-01-16T15:01:21.445916+0000","last_peered":"2024-01-16T15:01:21.445916+0000","last_clean":"2024-01-16T15:01:21.445916+0000","last_became_active":"2024-01-16T15:01:19.894743+0000","last_became_peered":"2024-01-16T15:01:19.894743+0000","last_unstale":"2024-01-16T15:01:21.445916+0000","last_undegraded":"2024-01-16T15:01:21.445916+0000","last_fullsized":"2024-01-16T15:01:21.445916+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:36:30.879279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.898017+0000","last_change":"2024-01-16T15:01:18.857052+0000","last_active":"2024-01-16T15:01:20.898017+0000","last_peered":"2024-01-16T15:01:20.898017+0000","last_clean":"2024-01-16T15:01:20.898017+0000","last_became_active":"2024-01-16T15:01:18.856769+0000","last_became_peered":"2024-01-16T15:01:18.856769+0000","last_unstale":"2024-01-16T15:01:20.898017+0000","last_undegraded":"2024-01-16T15:01:20.898017+0000","last_fullsized":"2024-01-16T15:01:20.898017+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:15:14.228212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.445168+0000","last_change":"2024-01-16T15:01:19.897898+0000","last_active":"2024-01-16T15:01:21.445168+0000","last_peered":"2024-01-16T15:01:21.445168+0000","last_clean":"2024-01-16T15:01:21.445168+0000","last_became_active":"2024-01-16T15:01:19.894918+0000","last_became_peered":"2024-01-16T15:01:19.894918+0000","last_unstale":"2024-01-16T15:01:21.445168+0000","last_undegraded":"2024-01-16T15:01:21.445168+0000","last_fullsized":"2024-01-16T15:01:21.445168+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:15:15.239596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.349565+0000","last_change":"2024-01-16T15:01:18.863397+0000","last_active":"2024-01-16T15:01:21.349565+0000","last_peered":"2024-01-16T15:01:21.349565+0000","last_clean":"2024-01-16T15:01:21.349565+0000","last_became_active":"2024-01-16T15:01:18.863134+0000","last_became_peered":"2024-01-16T15:01:18.863134+0000","last_unstale":"2024-01-16T15:01:21.349565+0000","last_undegraded":"2024-01-16T15:01:21.349565+0000","last_fullsized":"2024-01-16T15:01:21.349565+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:32:47.434130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.903501+0000","last_change":"2024-01-16T15:01:19.882788+0000","last_active":"2024-01-16T15:01:20.903501+0000","last_peered":"2024-01-16T15:01:20.903501+0000","last_clean":"2024-01-16T15:01:20.903501+0000","last_became_active":"2024-01-16T15:01:19.881913+0000","last_became_peered":"2024-01-16T15:01:19.881913+0000","last_unstale":"2024-01-16T15:01:20.903501+0000","last_undegraded":"2024-01-16T15:01:20.903501+0000","last_fullsized":"2024-01-16T15:01:20.903501+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:25:30.380602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.445009+0000","last_change":"2024-01-16T15:01:18.865837+0000","last_active":"2024-01-16T15:01:21.445009+0000","last_peered":"2024-01-16T15:01:21.445009+0000","last_clean":"2024-01-16T15:01:21.445009+0000","last_became_active":"2024-01-16T15:01:18.863538+0000","last_became_peered":"2024-01-16T15:01:18.863538+0000","last_unstale":"2024-01-16T15:01:21.445009+0000","last_undegraded":"2024-01-16T15:01:21.445009+0000","last_fullsized":"2024-01-16T15:01:21.445009+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:03:48.869753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.901091+0000","last_change":"2024-01-16T15:01:19.905185+0000","last_active":"2024-01-16T15:01:20.901091+0000","last_peered":"2024-01-16T15:01:20.901091+0000","last_clean":"2024-01-16T15:01:20.901091+0000","last_became_active":"2024-01-16T15:01:19.904821+0000","last_became_peered":"2024-01-16T15:01:19.904821+0000","last_unstale":"2024-01-16T15:01:20.901091+0000","last_undegraded":"2024-01-16T15:01:20.901091+0000","last_fullsized":"2024-01-16T15:01:20.901091+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:25:29.059223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.444699+0000","last_change":"2024-01-16T15:01:18.868539+0000","last_active":"2024-01-16T15:01:21.444699+0000","last_peered":"2024-01-16T15:01:21.444699+0000","last_clean":"2024-01-16T15:01:21.444699+0000","last_became_active":"2024-01-16T15:01:18.868205+0000","last_became_peered":"2024-01-16T15:01:18.868205+0000","last_unstale":"2024-01-16T15:01:21.444699+0000","last_undegraded":"2024-01-16T15:01:21.444699+0000","last_fullsized":"2024-01-16T15:01:21.444699+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:14:11.856450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.444082+0000","last_change":"2024-01-16T15:01:19.888191+0000","last_active":"2024-01-16T15:01:21.444082+0000","last_peered":"2024-01-16T15:01:21.444082+0000","last_clean":"2024-01-16T15:01:21.444082+0000","last_became_active":"2024-01-16T15:01:19.887906+0000","last_became_peered":"2024-01-16T15:01:19.887906+0000","last_unstale":"2024-01-16T15:01:21.444082+0000","last_undegraded":"2024-01-16T15:01:21.444082+0000","last_fullsized":"2024-01-16T15:01:21.444082+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:35:31.837292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.897495+0000","last_change":"2024-01-16T15:01:18.856602+0000","last_active":"2024-01-16T15:01:20.897495+0000","last_peered":"2024-01-16T15:01:20.897495+0000","last_clean":"2024-01-16T15:01:20.897495+0000","last_became_active":"2024-01-16T15:01:18.855724+0000","last_became_peered":"2024-01-16T15:01:18.855724+0000","last_unstale":"2024-01-16T15:01:20.897495+0000","last_undegraded":"2024-01-16T15:01:20.897495+0000","last_fullsized":"2024-01-16T15:01:20.897495+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:46:27.664271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.443381+0000","last_change":"2024-01-16T15:01:19.895075+0000","last_active":"2024-01-16T15:01:21.443381+0000","last_peered":"2024-01-16T15:01:21.443381+0000","last_clean":"2024-01-16T15:01:21.443381+0000","last_became_active":"2024-01-16T15:01:19.894819+0000","last_became_peered":"2024-01-16T15:01:19.894819+0000","last_unstale":"2024-01-16T15:01:21.443381+0000","last_undegraded":"2024-01-16T15:01:21.443381+0000","last_fullsized":"2024-01-16T15:01:21.443381+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:35:31.837292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.444283+0000","last_change":"2024-01-16T15:01:18.870847+0000","last_active":"2024-01-16T15:01:21.444283+0000","last_peered":"2024-01-16T15:01:21.444283+0000","last_clean":"2024-01-16T15:01:21.444283+0000","last_became_active":"2024-01-16T15:01:18.870559+0000","last_became_peered":"2024-01-16T15:01:18.870559+0000","last_unstale":"2024-01-16T15:01:21.444283+0000","last_undegraded":"2024-01-16T15:01:21.444283+0000","last_fullsized":"2024-01-16T15:01:21.444283+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:37:10.879045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.902097+0000","last_change":"2024-01-16T15:01:19.905944+0000","last_active":"2024-01-16T15:01:20.902097+0000","last_peered":"2024-01-16T15:01:20.902097+0000","last_clean":"2024-01-16T15:01:20.902097+0000","last_became_active":"2024-01-16T15:01:19.905659+0000","last_became_peered":"2024-01-16T15:01:19.905659+0000","last_unstale":"2024-01-16T15:01:20.902097+0000","last_undegraded":"2024-01-16T15:01:20.902097+0000","last_fullsized":"2024-01-16T15:01:20.902097+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:12:08.205887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.898329+0000","last_change":"2024-01-16T15:01:18.870502+0000","last_active":"2024-01-16T15:01:20.898329+0000","last_peered":"2024-01-16T15:01:20.898329+0000","last_clean":"2024-01-16T15:01:20.898329+0000","last_became_active":"2024-01-16T15:01:18.870044+0000","last_became_peered":"2024-01-16T15:01:18.870044+0000","last_unstale":"2024-01-16T15:01:20.898329+0000","last_undegraded":"2024-01-16T15:01:20.898329+0000","last_fullsized":"2024-01-16T15:01:20.898329+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:08:17.336503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.898272+0000","last_change":"2024-01-16T15:01:19.895553+0000","last_active":"2024-01-16T15:01:20.898272+0000","last_peered":"2024-01-16T15:01:20.898272+0000","last_clean":"2024-01-16T15:01:20.898272+0000","last_became_active":"2024-01-16T15:01:19.894475+0000","last_became_peered":"2024-01-16T15:01:19.894475+0000","last_unstale":"2024-01-16T15:01:20.898272+0000","last_undegraded":"2024-01-16T15:01:20.898272+0000","last_fullsized":"2024-01-16T15:01:20.898272+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:02:00.425436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.443073+0000","last_change":"2024-01-16T15:01:18.863763+0000","last_active":"2024-01-16T15:01:21.443073+0000","last_peered":"2024-01-16T15:01:21.443073+0000","last_clean":"2024-01-16T15:01:21.443073+0000","last_became_active":"2024-01-16T15:01:18.863388+0000","last_became_peered":"2024-01-16T15:01:18.863388+0000","last_unstale":"2024-01-16T15:01:21.443073+0000","last_undegraded":"2024-01-16T15:01:21.443073+0000","last_fullsized":"2024-01-16T15:01:21.443073+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:57:03.106778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.898124+0000","last_change":"2024-01-16T15:01:19.892719+0000","last_active":"2024-01-16T15:01:20.898124+0000","last_peered":"2024-01-16T15:01:20.898124+0000","last_clean":"2024-01-16T15:01:20.898124+0000","last_became_active":"2024-01-16T15:01:19.892442+0000","last_became_peered":"2024-01-16T15:01:19.892442+0000","last_unstale":"2024-01-16T15:01:20.898124+0000","last_undegraded":"2024-01-16T15:01:20.898124+0000","last_fullsized":"2024-01-16T15:01:20.898124+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:43:37.821973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.901985+0000","last_change":"2024-01-16T15:01:18.875621+0000","last_active":"2024-01-16T15:01:20.901985+0000","last_peered":"2024-01-16T15:01:20.901985+0000","last_clean":"2024-01-16T15:01:20.901985+0000","last_became_active":"2024-01-16T15:01:18.874654+0000","last_became_peered":"2024-01-16T15:01:18.874654+0000","last_unstale":"2024-01-16T15:01:20.901985+0000","last_undegraded":"2024-01-16T15:01:20.901985+0000","last_fullsized":"2024-01-16T15:01:20.901985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:14:11.856450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.901725+0000","last_change":"2024-01-16T15:01:19.905409+0000","last_active":"2024-01-16T15:01:20.901725+0000","last_peered":"2024-01-16T15:01:20.901725+0000","last_clean":"2024-01-16T15:01:20.901725+0000","last_became_active":"2024-01-16T15:01:19.905039+0000","last_became_peered":"2024-01-16T15:01:19.905039+0000","last_unstale":"2024-01-16T15:01:20.901725+0000","last_undegraded":"2024-01-16T15:01:20.901725+0000","last_fullsized":"2024-01-16T15:01:20.901725+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:15:15.239596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.898300+0000","last_change":"2024-01-16T15:01:18.862762+0000","last_active":"2024-01-16T15:01:20.898300+0000","last_peered":"2024-01-16T15:01:20.898300+0000","last_clean":"2024-01-16T15:01:20.898300+0000","last_became_active":"2024-01-16T15:01:18.862467+0000","last_became_peered":"2024-01-16T15:01:18.862467+0000","last_unstale":"2024-01-16T15:01:20.898300+0000","last_undegraded":"2024-01-16T15:01:20.898300+0000","last_fullsized":"2024-01-16T15:01:20.898300+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:25:28.047839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.443917+0000","last_change":"2024-01-16T15:01:19.893917+0000","last_active":"2024-01-16T15:01:21.443917+0000","last_peered":"2024-01-16T15:01:21.443917+0000","last_clean":"2024-01-16T15:01:21.443917+0000","last_became_active":"2024-01-16T15:01:19.893635+0000","last_became_peered":"2024-01-16T15:01:19.893635+0000","last_unstale":"2024-01-16T15:01:21.443917+0000","last_undegraded":"2024-01-16T15:01:21.443917+0000","last_fullsized":"2024-01-16T15:01:21.443917+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:46:28.675655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.445253+0000","last_change":"2024-01-16T15:01:18.872168+0000","last_active":"2024-01-16T15:01:21.445253+0000","last_peered":"2024-01-16T15:01:21.445253+0000","last_clean":"2024-01-16T15:01:21.445253+0000","last_became_active":"2024-01-16T15:01:18.871737+0000","last_became_peered":"2024-01-16T15:01:18.871737+0000","last_unstale":"2024-01-16T15:01:21.445253+0000","last_undegraded":"2024-01-16T15:01:21.445253+0000","last_fullsized":"2024-01-16T15:01:21.445253+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:57:03.106778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.897763+0000","last_change":"2024-01-16T15:01:19.906704+0000","last_active":"2024-01-16T15:01:20.897763+0000","last_peered":"2024-01-16T15:01:20.897763+0000","last_clean":"2024-01-16T15:01:20.897763+0000","last_became_active":"2024-01-16T15:01:19.906296+0000","last_became_peered":"2024-01-16T15:01:19.906296+0000","last_unstale":"2024-01-16T15:01:20.897763+0000","last_undegraded":"2024-01-16T15:01:20.897763+0000","last_fullsized":"2024-01-16T15:01:20.897763+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:12:08.205887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.443884+0000","last_change":"2024-01-16T15:01:18.871031+0000","last_active":"2024-01-16T15:01:21.443884+0000","last_peered":"2024-01-16T15:01:21.443884+0000","last_clean":"2024-01-16T15:01:21.443884+0000","last_became_active":"2024-01-16T15:01:18.870735+0000","last_became_peered":"2024-01-16T15:01:18.870735+0000","last_unstale":"2024-01-16T15:01:21.443884+0000","last_undegraded":"2024-01-16T15:01:21.443884+0000","last_fullsized":"2024-01-16T15:01:21.443884+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:11:58.134390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.902725+0000","last_change":"2024-01-16T15:01:19.884489+0000","last_active":"2024-01-16T15:01:20.902725+0000","last_peered":"2024-01-16T15:01:20.902725+0000","last_clean":"2024-01-16T15:01:20.902725+0000","last_became_active":"2024-01-16T15:01:19.884084+0000","last_became_peered":"2024-01-16T15:01:19.884084+0000","last_unstale":"2024-01-16T15:01:20.902725+0000","last_undegraded":"2024-01-16T15:01:20.902725+0000","last_fullsized":"2024-01-16T15:01:20.902725+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:31:08.299586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.897487+0000","last_change":"2024-01-16T15:01:18.865261+0000","last_active":"2024-01-16T15:01:20.897487+0000","last_peered":"2024-01-16T15:01:20.897487+0000","last_clean":"2024-01-16T15:01:20.897487+0000","last_became_active":"2024-01-16T15:01:18.864967+0000","last_became_peered":"2024-01-16T15:01:18.864967+0000","last_unstale":"2024-01-16T15:01:20.897487+0000","last_undegraded":"2024-01-16T15:01:20.897487+0000","last_fullsized":"2024-01-16T15:01:20.897487+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:30:41.519382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.444324+0000","last_change":"2024-01-16T15:01:19.895106+0000","last_active":"2024-01-16T15:01:21.444324+0000","last_peered":"2024-01-16T15:01:21.444324+0000","last_clean":"2024-01-16T15:01:21.444324+0000","last_became_active":"2024-01-16T15:01:19.892683+0000","last_became_peered":"2024-01-16T15:01:19.892683+0000","last_unstale":"2024-01-16T15:01:21.444324+0000","last_undegraded":"2024-01-16T15:01:21.444324+0000","last_fullsized":"2024-01-16T15:01:21.444324+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:41:22.704472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":20,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.897376+0000","last_change":"2024-01-16T15:01:18.863963+0000","last_active":"2024-01-16T15:01:20.897376+0000","last_peered":"2024-01-16T15:01:20.897376+0000","last_clean":"2024-01-16T15:01:20.897376+0000","last_became_active":"2024-01-16T15:01:18.861667+0000","last_became_peered":"2024-01-16T15:01:18.861667+0000","last_unstale":"2024-01-16T15:01:20.897376+0000","last_undegraded":"2024-01-16T15:01:20.897376+0000","last_fullsized":"2024-01-16T15:01:20.897376+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:14:11.856450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.446022+0000","last_change":"2024-01-16T15:01:19.904229+0000","last_active":"2024-01-16T15:01:21.446022+0000","last_peered":"2024-01-16T15:01:21.446022+0000","last_clean":"2024-01-16T15:01:21.446022+0000","last_became_active":"2024-01-16T15:01:19.903865+0000","last_became_peered":"2024-01-16T15:01:19.903865+0000","last_unstale":"2024-01-16T15:01:21.446022+0000","last_undegraded":"2024-01-16T15:01:21.446022+0000","last_fullsized":"2024-01-16T15:01:21.446022+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:25:29.059223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.444938+0000","last_change":"2024-01-16T15:01:18.860430+0000","last_active":"2024-01-16T15:01:21.444938+0000","last_peered":"2024-01-16T15:01:21.444938+0000","last_clean":"2024-01-16T15:01:21.444938+0000","last_became_active":"2024-01-16T15:01:18.860139+0000","last_became_peered":"2024-01-16T15:01:18.860139+0000","last_unstale":"2024-01-16T15:01:21.444938+0000","last_undegraded":"2024-01-16T15:01:21.444938+0000","last_fullsized":"2024-01-16T15:01:21.444938+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:41:21.693088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.902497+0000","last_change":"2024-01-16T15:01:19.901362+0000","last_active":"2024-01-16T15:01:20.902497+0000","last_peered":"2024-01-16T15:01:20.902497+0000","last_clean":"2024-01-16T15:01:20.902497+0000","last_became_active":"2024-01-16T15:01:19.900896+0000","last_became_peered":"2024-01-16T15:01:19.900896+0000","last_unstale":"2024-01-16T15:01:20.902497+0000","last_undegraded":"2024-01-16T15:01:20.902497+0000","last_fullsized":"2024-01-16T15:01:20.902497+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:47:30.255903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.445908+0000","last_change":"2024-01-16T15:01:18.872423+0000","last_active":"2024-01-16T15:01:21.445908+0000","last_peered":"2024-01-16T15:01:21.445908+0000","last_clean":"2024-01-16T15:01:21.445908+0000","last_became_active":"2024-01-16T15:01:18.872040+0000","last_became_peered":"2024-01-16T15:01:18.872040+0000","last_unstale":"2024-01-16T15:01:21.445908+0000","last_undegraded":"2024-01-16T15:01:21.445908+0000","last_fullsized":"2024-01-16T15:01:21.445908+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:11:58.134390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.444642+0000","last_change":"2024-01-16T15:01:19.902396+0000","last_active":"2024-01-16T15:01:21.444642+0000","last_peered":"2024-01-16T15:01:21.444642+0000","last_clean":"2024-01-16T15:01:21.444642+0000","last_became_active":"2024-01-16T15:01:19.902052+0000","last_became_peered":"2024-01-16T15:01:19.902052+0000","last_unstale":"2024-01-16T15:01:21.444642+0000","last_undegraded":"2024-01-16T15:01:21.444642+0000","last_fullsized":"2024-01-16T15:01:21.444642+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:36:30.879279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.446228+0000","last_change":"2024-01-16T15:01:18.862214+0000","last_active":"2024-01-16T15:01:21.446228+0000","last_peered":"2024-01-16T15:01:21.446228+0000","last_clean":"2024-01-16T15:01:21.446228+0000","last_became_active":"2024-01-16T15:01:18.861928+0000","last_became_peered":"2024-01-16T15:01:18.861928+0000","last_unstale":"2024-01-16T15:01:21.446228+0000","last_undegraded":"2024-01-16T15:01:21.446228+0000","last_fullsized":"2024-01-16T15:01:21.446228+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:03:56.456093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.349806+0000","last_change":"2024-01-16T15:01:19.894249+0000","last_active":"2024-01-16T15:01:21.349806+0000","last_peered":"2024-01-16T15:01:21.349806+0000","last_clean":"2024-01-16T15:01:21.349806+0000","last_became_active":"2024-01-16T15:01:19.893879+0000","last_became_peered":"2024-01-16T15:01:19.893879+0000","last_unstale":"2024-01-16T15:01:21.349806+0000","last_undegraded":"2024-01-16T15:01:21.349806+0000","last_fullsized":"2024-01-16T15:01:21.349806+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:12:29.462524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.446140+0000","last_change":"2024-01-16T15:01:18.868277+0000","last_active":"2024-01-16T15:01:21.446140+0000","last_peered":"2024-01-16T15:01:21.446140+0000","last_clean":"2024-01-16T15:01:21.446140+0000","last_became_active":"2024-01-16T15:01:18.867949+0000","last_became_peered":"2024-01-16T15:01:18.867949+0000","last_unstale":"2024-01-16T15:01:21.446140+0000","last_undegraded":"2024-01-16T15:01:21.446140+0000","last_fullsized":"2024-01-16T15:01:21.446140+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:41:21.693088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.349192+0000","last_change":"2024-01-16T15:01:19.886196+0000","last_active":"2024-01-16T15:01:21.349192+0000","last_peered":"2024-01-16T15:01:21.349192+0000","last_clean":"2024-01-16T15:01:21.349192+0000","last_became_active":"2024-01-16T15:01:19.885784+0000","last_became_peered":"2024-01-16T15:01:19.885784+0000","last_unstale":"2024-01-16T15:01:21.349192+0000","last_undegraded":"2024-01-16T15:01:21.349192+0000","last_fullsized":"2024-01-16T15:01:21.349192+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:06:45.119287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.442895+0000","last_change":"2024-01-16T15:01:18.864458+0000","last_active":"2024-01-16T15:01:21.442895+0000","last_peered":"2024-01-16T15:01:21.442895+0000","last_clean":"2024-01-16T15:01:21.442895+0000","last_became_active":"2024-01-16T15:01:18.864143+0000","last_became_peered":"2024-01-16T15:01:18.864143+0000","last_unstale":"2024-01-16T15:01:21.442895+0000","last_undegraded":"2024-01-16T15:01:21.442895+0000","last_fullsized":"2024-01-16T15:01:21.442895+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:03:56.456093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.349731+0000","last_change":"2024-01-16T15:01:19.899007+0000","last_active":"2024-01-16T15:01:21.349731+0000","last_peered":"2024-01-16T15:01:21.349731+0000","last_clean":"2024-01-16T15:01:21.349731+0000","last_became_active":"2024-01-16T15:01:19.898356+0000","last_became_peered":"2024-01-16T15:01:19.898356+0000","last_unstale":"2024-01-16T15:01:21.349731+0000","last_undegraded":"2024-01-16T15:01:21.349731+0000","last_fullsized":"2024-01-16T15:01:21.349731+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:09:03.014818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.445307+0000","last_change":"2024-01-16T15:01:18.875155+0000","last_active":"2024-01-16T15:01:21.445307+0000","last_peered":"2024-01-16T15:01:21.445307+0000","last_clean":"2024-01-16T15:01:21.445307+0000","last_became_active":"2024-01-16T15:01:18.874825+0000","last_became_peered":"2024-01-16T15:01:18.874825+0000","last_unstale":"2024-01-16T15:01:21.445307+0000","last_undegraded":"2024-01-16T15:01:21.445307+0000","last_fullsized":"2024-01-16T15:01:21.445307+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:57:03.106778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.349305+0000","last_change":"2024-01-16T15:01:19.897044+0000","last_active":"2024-01-16T15:01:21.349305+0000","last_peered":"2024-01-16T15:01:21.349305+0000","last_clean":"2024-01-16T15:01:21.349305+0000","last_became_active":"2024-01-16T15:01:19.896691+0000","last_became_peered":"2024-01-16T15:01:19.896691+0000","last_unstale":"2024-01-16T15:01:21.349305+0000","last_undegraded":"2024-01-16T15:01:21.349305+0000","last_fullsized":"2024-01-16T15:01:21.349305+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:22:11.277546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.444099+0000","last_change":"2024-01-16T15:01:18.862881+0000","last_active":"2024-01-16T15:01:21.444099+0000","last_peered":"2024-01-16T15:01:21.444099+0000","last_clean":"2024-01-16T15:01:21.444099+0000","last_became_active":"2024-01-16T15:01:18.862433+0000","last_became_peered":"2024-01-16T15:01:18.862433+0000","last_unstale":"2024-01-16T15:01:21.444099+0000","last_undegraded":"2024-01-16T15:01:21.444099+0000","last_fullsized":"2024-01-16T15:01:21.444099+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:54:04.021554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.903202+0000","last_change":"2024-01-16T15:01:19.891593+0000","last_active":"2024-01-16T15:01:20.903202+0000","last_peered":"2024-01-16T15:01:20.903202+0000","last_clean":"2024-01-16T15:01:20.903202+0000","last_became_active":"2024-01-16T15:01:19.890549+0000","last_became_peered":"2024-01-16T15:01:19.890549+0000","last_unstale":"2024-01-16T15:01:20.903202+0000","last_undegraded":"2024-01-16T15:01:20.903202+0000","last_fullsized":"2024-01-16T15:01:20.903202+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:52:44.965933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.900696+0000","last_change":"2024-01-16T15:01:18.876724+0000","last_active":"2024-01-16T15:01:20.900696+0000","last_peered":"2024-01-16T15:01:20.900696+0000","last_clean":"2024-01-16T15:01:20.900696+0000","last_became_active":"2024-01-16T15:01:18.875910+0000","last_became_peered":"2024-01-16T15:01:18.875910+0000","last_unstale":"2024-01-16T15:01:20.900696+0000","last_undegraded":"2024-01-16T15:01:20.900696+0000","last_fullsized":"2024-01-16T15:01:20.900696+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:12:28.451140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.349664+0000","last_change":"2024-01-16T15:01:19.904235+0000","last_active":"2024-01-16T15:01:21.349664+0000","last_peered":"2024-01-16T15:01:21.349664+0000","last_clean":"2024-01-16T15:01:21.349664+0000","last_became_active":"2024-01-16T15:01:19.903591+0000","last_became_peered":"2024-01-16T15:01:19.903591+0000","last_unstale":"2024-01-16T15:01:21.349664+0000","last_undegraded":"2024-01-16T15:01:21.349664+0000","last_fullsized":"2024-01-16T15:01:21.349664+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:54:05.032938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.445023+0000","last_change":"2024-01-16T15:01:18.876461+0000","last_active":"2024-01-16T15:01:21.445023+0000","last_peered":"2024-01-16T15:01:21.445023+0000","last_clean":"2024-01-16T15:01:21.445023+0000","last_became_active":"2024-01-16T15:01:18.876220+0000","last_became_peered":"2024-01-16T15:01:18.876220+0000","last_unstale":"2024-01-16T15:01:21.445023+0000","last_undegraded":"2024-01-16T15:01:21.445023+0000","last_fullsized":"2024-01-16T15:01:21.445023+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:14:11.856450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.902491+0000","last_change":"2024-01-16T15:01:19.904111+0000","last_active":"2024-01-16T15:01:20.902491+0000","last_peered":"2024-01-16T15:01:20.902491+0000","last_clean":"2024-01-16T15:01:20.902491+0000","last_became_active":"2024-01-16T15:01:19.903825+0000","last_became_peered":"2024-01-16T15:01:19.903825+0000","last_unstale":"2024-01-16T15:01:20.902491+0000","last_undegraded":"2024-01-16T15:01:20.902491+0000","last_fullsized":"2024-01-16T15:01:20.902491+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:52:44.965933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.443185+0000","last_change":"2024-01-16T15:01:18.865194+0000","last_active":"2024-01-16T15:01:21.443185+0000","last_peered":"2024-01-16T15:01:21.443185+0000","last_clean":"2024-01-16T15:01:21.443185+0000","last_became_active":"2024-01-16T15:01:18.864944+0000","last_became_peered":"2024-01-16T15:01:18.864944+0000","last_unstale":"2024-01-16T15:01:21.443185+0000","last_undegraded":"2024-01-16T15:01:21.443185+0000","last_fullsized":"2024-01-16T15:01:21.443185+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:22:02.623082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.442573+0000","last_change":"2024-01-16T15:01:19.910243+0000","last_active":"2024-01-16T15:01:21.442573+0000","last_peered":"2024-01-16T15:01:21.442573+0000","last_clean":"2024-01-16T15:01:21.442573+0000","last_became_active":"2024-01-16T15:01:19.909971+0000","last_became_peered":"2024-01-16T15:01:19.909971+0000","last_unstale":"2024-01-16T15:01:21.442573+0000","last_undegraded":"2024-01-16T15:01:21.442573+0000","last_fullsized":"2024-01-16T15:01:21.442573+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:36:47.158317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.444713+0000","last_change":"2024-01-16T15:01:18.861635+0000","last_active":"2024-01-16T15:01:21.444713+0000","last_peered":"2024-01-16T15:01:21.444713+0000","last_clean":"2024-01-16T15:01:21.444713+0000","last_became_active":"2024-01-16T15:01:18.859919+0000","last_became_peered":"2024-01-16T15:01:18.859919+0000","last_unstale":"2024-01-16T15:01:21.444713+0000","last_undegraded":"2024-01-16T15:01:21.444713+0000","last_fullsized":"2024-01-16T15:01:21.444713+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:22:02.623082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.444884+0000","last_change":"2024-01-16T15:01:19.898274+0000","last_active":"2024-01-16T15:01:21.444884+0000","last_peered":"2024-01-16T15:01:21.444884+0000","last_clean":"2024-01-16T15:01:21.444884+0000","last_became_active":"2024-01-16T15:01:19.898025+0000","last_became_peered":"2024-01-16T15:01:19.898025+0000","last_unstale":"2024-01-16T15:01:21.444884+0000","last_undegraded":"2024-01-16T15:01:21.444884+0000","last_fullsized":"2024-01-16T15:01:21.444884+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:14:12.867834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.443134+0000","last_change":"2024-01-16T15:01:18.871497+0000","last_active":"2024-01-16T15:01:21.443134+0000","last_peered":"2024-01-16T15:01:21.443134+0000","last_clean":"2024-01-16T15:01:21.443134+0000","last_became_active":"2024-01-16T15:01:18.871066+0000","last_became_peered":"2024-01-16T15:01:18.871066+0000","last_unstale":"2024-01-16T15:01:21.443134+0000","last_undegraded":"2024-01-16T15:01:21.443134+0000","last_fullsized":"2024-01-16T15:01:21.443134+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:12:28.451140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.442879+0000","last_change":"2024-01-16T15:01:19.891355+0000","last_active":"2024-01-16T15:01:21.442879+0000","last_peered":"2024-01-16T15:01:21.442879+0000","last_clean":"2024-01-16T15:01:21.442879+0000","last_became_active":"2024-01-16T15:01:19.891098+0000","last_became_peered":"2024-01-16T15:01:19.891098+0000","last_unstale":"2024-01-16T15:01:21.442879+0000","last_undegraded":"2024-01-16T15:01:21.442879+0000","last_fullsized":"2024-01-16T15:01:21.442879+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:35:31.837292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.444102+0000","last_change":"2024-01-16T15:01:18.871366+0000","last_active":"2024-01-16T15:01:21.444102+0000","last_peered":"2024-01-16T15:01:21.444102+0000","last_clean":"2024-01-16T15:01:21.444102+0000","last_became_active":"2024-01-16T15:01:18.870276+0000","last_became_peered":"2024-01-16T15:01:18.870276+0000","last_unstale":"2024-01-16T15:01:21.444102+0000","last_undegraded":"2024-01-16T15:01:21.444102+0000","last_fullsized":"2024-01-16T15:01:21.444102+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:57:03.106778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.897428+0000","last_change":"2024-01-16T15:01:19.903450+0000","last_active":"2024-01-16T15:01:20.897428+0000","last_peered":"2024-01-16T15:01:20.897428+0000","last_clean":"2024-01-16T15:01:20.897428+0000","last_became_active":"2024-01-16T15:01:19.903156+0000","last_became_peered":"2024-01-16T15:01:19.903156+0000","last_unstale":"2024-01-16T15:01:20.897428+0000","last_undegraded":"2024-01-16T15:01:20.897428+0000","last_fullsized":"2024-01-16T15:01:20.897428+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:31:08.299586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.444176+0000","last_change":"2024-01-16T15:01:18.872671+0000","last_active":"2024-01-16T15:01:21.444176+0000","last_peered":"2024-01-16T15:01:21.444176+0000","last_clean":"2024-01-16T15:01:21.444176+0000","last_became_active":"2024-01-16T15:01:18.872362+0000","last_became_peered":"2024-01-16T15:01:18.872362+0000","last_unstale":"2024-01-16T15:01:21.444176+0000","last_undegraded":"2024-01-16T15:01:21.444176+0000","last_fullsized":"2024-01-16T15:01:21.444176+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:12:28.451140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.443953+0000","last_change":"2024-01-16T15:01:19.890285+0000","last_active":"2024-01-16T15:01:21.443953+0000","last_peered":"2024-01-16T15:01:21.443953+0000","last_clean":"2024-01-16T15:01:21.443953+0000","last_became_active":"2024-01-16T15:01:19.889891+0000","last_became_peered":"2024-01-16T15:01:19.889891+0000","last_unstale":"2024-01-16T15:01:21.443953+0000","last_undegraded":"2024-01-16T15:01:21.443953+0000","last_fullsized":"2024-01-16T15:01:21.443953+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:36:47.158317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.897514+0000","last_change":"2024-01-16T15:01:18.860505+0000","last_active":"2024-01-16T15:01:20.897514+0000","last_peered":"2024-01-16T15:01:20.897514+0000","last_clean":"2024-01-16T15:01:20.897514+0000","last_became_active":"2024-01-16T15:01:18.860222+0000","last_became_peered":"2024-01-16T15:01:18.860222+0000","last_unstale":"2024-01-16T15:01:20.897514+0000","last_undegraded":"2024-01-16T15:01:20.897514+0000","last_fullsized":"2024-01-16T15:01:20.897514+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:36:46.146932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.903470+0000","last_change":"2024-01-16T15:01:19.896174+0000","last_active":"2024-01-16T15:01:20.903470+0000","last_peered":"2024-01-16T15:01:20.903470+0000","last_clean":"2024-01-16T15:01:20.903470+0000","last_became_active":"2024-01-16T15:01:19.894009+0000","last_became_peered":"2024-01-16T15:01:19.894009+0000","last_unstale":"2024-01-16T15:01:20.903470+0000","last_undegraded":"2024-01-16T15:01:20.903470+0000","last_fullsized":"2024-01-16T15:01:20.903470+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:43:37.821973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.444478+0000","last_change":"2024-01-16T15:01:18.876295+0000","last_active":"2024-01-16T15:01:21.444478+0000","last_peered":"2024-01-16T15:01:21.444478+0000","last_clean":"2024-01-16T15:01:21.444478+0000","last_became_active":"2024-01-16T15:01:18.875885+0000","last_became_peered":"2024-01-16T15:01:18.875885+0000","last_unstale":"2024-01-16T15:01:21.444478+0000","last_undegraded":"2024-01-16T15:01:21.444478+0000","last_fullsized":"2024-01-16T15:01:21.444478+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:03:48.869753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":179,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-16T15:01:19.870231+0000","last_change":"2024-01-16T14:59:53.430715+0000","last_active":"2024-01-16T15:01:19.870231+0000","last_peered":"2024-01-16T15:01:19.870231+0000","last_clean":"2024-01-16T15:01:19.870231+0000","last_became_active":"2024-01-16T14:59:53.428209+0000","last_became_peered":"2024-01-16T14:59:53.428209+0000","last_unstale":"2024-01-16T15:01:19.870231+0000","last_undegraded":"2024-01-16T15:01:19.870231+0000","last_fullsized":"2024-01-16T15:01:19.870231+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:59:52.011232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:59:52.011232+0000","last_clean_scrub_stamp":"2024-01-16T14:59:52.011232+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:57:42.594848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":983040,"data_stored":742357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 16 15:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":1.3280000000000001}]},{"osd":1,"last update":"Tue Jan 16 15:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.724,"5min":1.724,"15min":1.724},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.143,"5min":1.143,"15min":1.143},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":0.96599999999999997}]},{"osd":2,"last update":"Tue Jan 16 15:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.893,"5min":1.893,"15min":1.893},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.21,"5min":1.21,"15min":1.21},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.2110000000000001}]},{"osd":3,"last update":"Tue Jan 16 15:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"min":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.399,"5min":1.399,"15min":1.399},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.1339999999999999}]},{"osd":4,"last update":"Tue Jan 16 15:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.4889999999999999,"5min":2.4889999999999999,"15min":2.4889999999999999},"last":1.6439999999999999},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.5259999999999998,"5min":2.5259999999999998,"15min":2.5259999999999998},"last":1.409}]},{"osd":5,"last update":"Tue Jan 16 15:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.29,"15min":1.29},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.581,"5min":2.581,"15min":2.581},"last":1.716}]},{"osd":6,"last update":"Tue Jan 16 15:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1.522,"5min":1.522,"15min":1.522},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.3740000000000001,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":1.766},{"interface":"front","average":{"1min":1.415,"5min":1.415,"15min":1.415},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.0939999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.5800000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":146215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.0129999999999999}]},{"osd":1,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":1.236},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.4449999999999998,"5min":2.4449999999999998,"15min":2.4449999999999998},"last":1.4630000000000001}]},{"osd":2,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":3.0009999999999999,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":1.599},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.488,"5min":2.488,"15min":2.488},"last":1.1459999999999999}]},{"osd":3,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.373,"15min":1.373},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":4.0389999999999997,"5min":4.0389999999999997,"15min":4.0389999999999997},"last":1.514},{"interface":"front","average":{"1min":1.397,"5min":1.397,"15min":1.397},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":3.649,"5min":3.649,"15min":3.649},"last":1.427}]},{"osd":4,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.6720000000000002,"5min":2.6720000000000002,"15min":2.6720000000000002},"last":1.333},{"interface":"front","average":{"1min":1.54,"5min":1.54,"15min":1.54},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":3.7719999999999998,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":1.2849999999999999}]},{"osd":5,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":4.3840000000000003,"5min":4.3840000000000003,"15min":4.3840000000000003},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":3.3559999999999999,"5min":3.3559999999999999,"15min":3.3559999999999999},"last":1.669}]},{"osd":7,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":2.8479999999999999,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":4.2729999999999997,"5min":4.2729999999999997,"15min":4.2729999999999997},"last":1.5600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":146271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":1.026},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.80600000000000005}]},{"osd":1,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":1.6719999999999999}]},{"osd":2,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.238},{"interface":"front","average":{"1min":1.294,"5min":1.294,"15min":1.294},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.968,"5min":2.968,"15min":2.968},"last":1.508}]},{"osd":3,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":11.638,"5min":11.638,"15min":11.638},"last":1.482},{"interface":"front","average":{"1min":1.857,"5min":1.857,"15min":1.857},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":11.901999999999999,"5min":11.901999999999999,"15min":11.901999999999999},"last":1.452}]},{"osd":4,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.266,"5min":2.266,"15min":2.266},"last":1.4139999999999999}]},{"osd":6,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.7030000000000001},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.0950000000000002,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":1.5429999999999999}]},{"osd":7,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":2.3679999999999999,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":1.6020000000000001},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":2.5640000000000001,"5min":2.5640000000000001,"15min":2.5640000000000001},"last":1.639}]}]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":145703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 16 15:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.5489999999999999},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":1.0469999999999999}]},{"osd":1,"last update":"Tue Jan 16 15:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":3.0049999999999999,"5min":3.0049999999999999,"15min":3.0049999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.22,"5min":1.22,"15min":1.22},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.9470000000000001,"5min":2.9470000000000001,"15min":2.9470000000000001},"last":1.494}]},{"osd":2,"last update":"Tue Jan 16 15:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":2.137,"5min":2.137,"15min":2.137},"last":0.999},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":2.2570000000000001,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":1.373}]},{"osd":3,"last update":"Tue Jan 16 15:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":1.155},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.2250000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.28}]},{"osd":5,"last update":"Tue Jan 16 15:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.496,"15min":1.496},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":2.4020000000000001,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.454,"5min":1.454,"15min":1.454},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":1.329}]},{"osd":6,"last update":"Tue Jan 16 15:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":2.4529999999999998,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.496,"5min":2.496,"15min":2.496},"last":1.4450000000000001}]},{"osd":7,"last update":"Tue Jan 16 15:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.1960000000000002,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"min":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":2.536,"5min":2.536,"15min":2.536},"last":1.405}]}]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":153533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 16 15:00:53 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.909,"5min":1.909,"15min":1.909},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":2.863,"5min":2.863,"15min":2.863},"last":1.0029999999999999}]},{"osd":1,"last update":"Tue Jan 16 15:00:53 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.956,"5min":1.956,"15min":1.956},"last":1.51}]},{"osd":2,"last update":"Tue Jan 16 15:00:53 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.4670000000000001,"5min":2.4670000000000001,"15min":2.4670000000000001},"last":1.095},{"interface":"front","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":3.8530000000000002,"5min":3.8530000000000002,"15min":3.8530000000000002},"last":1.6850000000000001}]},{"osd":4,"last update":"Tue Jan 16 15:00:53 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":2.9950000000000001,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":1.272},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":2.6899999999999999,"5min":2.6899999999999999,"15min":2.6899999999999999},"last":1.2150000000000001}]},{"osd":5,"last update":"Tue Jan 16 15:00:53 2024","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.6509999999999998,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.8260000000000001,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":1.601}]},{"osd":6,"last update":"Tue Jan 16 15:00:53 2024","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.393,"15min":1.393},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.952,"5min":2.952,"15min":2.952},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.726,"5min":2.726,"15min":2.726},"last":1.73}]},{"osd":7,"last update":"Tue Jan 16 15:00:53 2024","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.508,"15min":1.508},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":3.9529999999999998,"5min":3.9529999999999998,"15min":3.9529999999999998},"last":2.3039999999999998},{"interface":"front","average":{"1min":1.498,"5min":1.498,"15min":1.498},"min":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"max":{"1min":2.911,"5min":2.911,"15min":2.911},"last":1.6419999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":151969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.753},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.208}]},{"osd":1,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.90200000000000002}]},{"osd":3,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":1.056},{"interface":"front","average":{"1min":1.232,"5min":1.232,"15min":1.232},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.155}]},{"osd":4,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.24,"15min":1.24},"min":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.296}]},{"osd":5,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.542,"5min":1.542,"15min":1.542},"min":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"max":{"1min":3.5030000000000001,"5min":3.5030000000000001,"15min":3.5030000000000001},"last":1.26},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"min":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":2.758,"5min":2.758,"15min":2.758},"last":1.6910000000000001}]},{"osd":6,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.476,"15min":1.476},"min":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"max":{"1min":2.1219999999999999,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.516,"5min":1.516,"15min":1.516},"min":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"max":{"1min":2.0030000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":1.4730000000000001}]},{"osd":7,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"min":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"max":{"1min":2.8620000000000001,"5min":2.8620000000000001,"15min":2.8620000000000001},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":2.2069999999999999,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":1.4319999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":147233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 16 15:00:53 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":2.5139999999999998,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":1.425},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":2.8999999999999999,"5min":2.8999999999999999,"15min":2.8999999999999999},"last":0.80000000000000004}]},{"osd":2,"last update":"Tue Jan 16 15:00:53 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.024,"5min":2.024,"15min":2.024},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":1.284}]},{"osd":3,"last update":"Tue Jan 16 15:00:53 2024","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.5019999999999998,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":5.4210000000000003,"5min":5.4210000000000003,"15min":5.4210000000000003},"last":1.202}]},{"osd":4,"last update":"Tue Jan 16 15:00:53 2024","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.177},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.4319999999999999,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":1.546}]},{"osd":5,"last update":"Tue Jan 16 15:00:53 2024","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.431,"15min":1.431},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":5.3819999999999997,"5min":5.3819999999999997,"15min":5.3819999999999997},"last":1.371},{"interface":"front","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":2.1640000000000001,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.333}]},{"osd":6,"last update":"Tue Jan 16 15:00:53 2024","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.28,"15min":1.28},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":2.2570000000000001,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":1.254}]},{"osd":7,"last update":"Tue Jan 16 15:00:53 2024","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.599},{"interface":"front","average":{"1min":1.294,"5min":1.294,"15min":1.294},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.3399999999999999,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.091}]}]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":974848,"data_stored":741248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Jan 16 15:00:55 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.5110000000000001,"5min":2.5110000000000001,"15min":2.5110000000000001},"last":1.339}]},{"osd":2,"last update":"Tue Jan 16 15:00:55 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":1.0620000000000001},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":0.61099999999999999}]},{"osd":3,"last update":"Tue Jan 16 15:00:55 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.008},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":0.96299999999999997}]},{"osd":4,"last update":"Tue Jan 16 15:00:55 2024","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.149,"15min":1.149},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.4660000000000002,"5min":2.4660000000000002,"15min":2.4660000000000002},"last":1.194},{"interface":"front","average":{"1min":1.242,"5min":1.242,"15min":1.242},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.1160000000000001,"5min":2.1160000000000001,"15min":2.1160000000000001},"last":1.4870000000000001}]},{"osd":5,"last update":"Tue Jan 16 15:00:55 2024","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.3740000000000001,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":3.23,"5min":3.23,"15min":3.23},"last":1.375}]},{"osd":6,"last update":"Tue Jan 16 15:00:55 2024","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.3580000000000001,"5min":2.3580000000000001,"15min":2.3580000000000001},"last":1.399},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":3.2890000000000001,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":1.423}]},{"osd":7,"last update":"Tue Jan 16 15:00:55 2024","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.329,"15min":1.329},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":3.3500000000000001,"5min":3.3500000000000001,"15min":3.3500000000000001},"last":1.29},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":2.621,"5min":2.621,"15min":2.621},"last":1.306}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-16T15:01:24.269 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-16T15:01:24.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.532+0000 7f87dcc00640 1 Processor -- start 2024-01-16T15:01:24.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.532+0000 7f87dcc00640 1 -- start start 2024-01-16T15:01:24.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.534+0000 7f87dcc00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f87d80cfb60 0x7f87d8161df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:24.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.534+0000 7f87dcc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87d815dc80 0x7f87d815e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:24.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.534+0000 7f87dcc00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87d80ce0b0 con 0x7f87d8157c70 2024-01-16T15:01:24.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.534+0000 7f87dcc00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87d8161960 con 0x7f87d80cfb60 2024-01-16T15:01:24.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.534+0000 7f87dcc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d815e670 con 0x7f87d815dc80 2024-01-16T15:01:24.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.534+0000 7f87cfe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f87d80cfb60 0x7f87d8161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:24.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.534+0000 7f87cfe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f87d80cfb60 0x7f87d8161df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59226/0 (socket says 172.21.15.143:59226) 2024-01-16T15:01:24.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.534+0000 7f87cfe00640 1 -- 172.21.15.143:0/1699743831 learned_addr learned my addr 172.21.15.143:0/1699743831 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:24.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.534+0000 7f87d6800640 1 --2- 172.21.15.143:0/1699743831 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87d815dc80 0x7f87d815e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:24.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.535+0000 7f87cfe00640 1 -- 172.21.15.143:0/1699743831 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87d815dc80 msgr2=0x7f87d815e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:24.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.535+0000 7f87cfe00640 1 --2- 172.21.15.143:0/1699743831 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87d815dc80 0x7f87d815e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:24.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.535+0000 7f87cfe00640 1 -- 172.21.15.143:0/1699743831 >> v1:172.21.15.204:6789/0 conn(0x7f87d8157c70 legacy=0x7f87d80cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:24.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.535+0000 7f87cfe00640 1 -- 172.21.15.143:0/1699743831 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d815e8d0 con 0x7f87d80cfb60 2024-01-16T15:01:24.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.536+0000 7f87cfe00640 1 --2- 172.21.15.143:0/1699743831 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87d80cfb60 0x7f87d8161df0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f87c0009870 tx=0x7f87c002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=921cf8598f77f7ca server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:24.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.536+0000 7f87d5400640 1 -- 172.21.15.143:0/1699743831 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c0031640 con 0x7f87d80cfb60 2024-01-16T15:01:24.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.536+0000 7f87d5400640 1 -- 172.21.15.143:0/1699743831 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c00317a0 con 0x7f87d80cfb60 2024-01-16T15:01:24.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.536+0000 7f87d5400640 1 -- 172.21.15.143:0/1699743831 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c0031b30 con 0x7f87d80cfb60 2024-01-16T15:01:24.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.537+0000 7f87dcc00640 1 -- 172.21.15.143:0/1699743831 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87d80cfb60 msgr2=0x7f87d8161df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:24.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.537+0000 7f87dcc00640 1 --2- 172.21.15.143:0/1699743831 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87d80cfb60 0x7f87d8161df0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f87c0009870 tx=0x7f87c002b8f0 comp rx=0 tx=0).stop 2024-01-16T15:01:24.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.537+0000 7f87dcc00640 1 -- 172.21.15.143:0/1699743831 shutdown_connections 2024-01-16T15:01:24.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.537+0000 7f87dcc00640 1 --2- 172.21.15.143:0/1699743831 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87d815dc80 0x7f87d815e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:24.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.537+0000 7f87dcc00640 1 --2- 172.21.15.143:0/1699743831 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87d80cfb60 0x7f87d8161df0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:24.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.537+0000 7f87dcc00640 1 -- 172.21.15.143:0/1699743831 >> 172.21.15.143:0/1699743831 conn(0x7f87d806a920 msgr2=0x7f87d806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:24.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.537+0000 7f87dcc00640 1 -- 172.21.15.143:0/1699743831 shutdown_connections 2024-01-16T15:01:24.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.537+0000 7f87dcc00640 1 -- 172.21.15.143:0/1699743831 wait complete. 2024-01-16T15:01:24.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.538+0000 7f87dcc00640 1 Processor -- start 2024-01-16T15:01:24.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.539+0000 7f87dcc00640 1 -- start start 2024-01-16T15:01:24.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.539+0000 7f87dcc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87d80cfb60 0x7f87d8151e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:24.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.539+0000 7f87dcc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87d8157c70 0x7f87d809bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:24.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.539+0000 7f87dcc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87d815dc80 0x7f87d809c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:24.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.540+0000 7f87dcc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d81f44a0 con 0x7f87d8157c70 2024-01-16T15:01:24.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.540+0000 7f87dcc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d80cbd00 con 0x7f87d80cfb60 2024-01-16T15:01:24.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.540+0000 7f87dcc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d80cb180 con 0x7f87d815dc80 2024-01-16T15:01:24.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.540+0000 7f87cfe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87d8157c70 0x7f87d809bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:24.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.540+0000 7f87cfe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87d8157c70 0x7f87d809bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59228/0 (socket says 172.21.15.143:59228) 2024-01-16T15:01:24.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.540+0000 7f87cfe00640 1 -- 172.21.15.143:0/2682005051 learned_addr learned my addr 172.21.15.143:0/2682005051 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:24.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.540+0000 7f87d6800640 1 --2- 172.21.15.143:0/2682005051 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87d815dc80 0x7f87d809c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:24.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.540+0000 7f87d5e00640 1 --2- 172.21.15.143:0/2682005051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87d80cfb60 0x7f87d8151e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:24.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.540+0000 7f87cfe00640 1 -- 172.21.15.143:0/2682005051 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87d815dc80 msgr2=0x7f87d809c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:24.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.540+0000 7f87cfe00640 1 --2- 172.21.15.143:0/2682005051 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87d815dc80 0x7f87d809c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:24.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.540+0000 7f87cfe00640 1 -- 172.21.15.143:0/2682005051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87d80cfb60 msgr2=0x7f87d8151e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:24.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.540+0000 7f87cfe00640 1 --2- 172.21.15.143:0/2682005051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87d80cfb60 0x7f87d8151e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:24.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.540+0000 7f87cfe00640 1 -- 172.21.15.143:0/2682005051 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c00094d0 con 0x7f87d8157c70 2024-01-16T15:01:24.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.541+0000 7f87cfe00640 1 --2- 172.21.15.143:0/2682005051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87d8157c70 0x7f87d809bb60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f87c0000c00 tx=0x7f87c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:24.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.541+0000 7f87ce000640 1 -- 172.21.15.143:0/2682005051 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c003ac60 con 0x7f87d8157c70 2024-01-16T15:01:24.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.541+0000 7f87ce000640 1 -- 172.21.15.143:0/2682005051 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c003adc0 con 0x7f87d8157c70 2024-01-16T15:01:24.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.541+0000 7f87ce000640 1 -- 172.21.15.143:0/2682005051 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c0044630 con 0x7f87d8157c70 2024-01-16T15:01:24.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.541+0000 7f87dcc00640 1 -- 172.21.15.143:0/2682005051 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87d80cb460 con 0x7f87d8157c70 2024-01-16T15:01:24.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.541+0000 7f87dcc00640 1 -- 172.21.15.143:0/2682005051 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87d80cba20 con 0x7f87d8157c70 2024-01-16T15:01:24.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.543+0000 7f87dcc00640 1 -- 172.21.15.143:0/2682005051 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87d8067470 con 0x7f87d8157c70 2024-01-16T15:01:24.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.543+0000 7f87ce000640 1 -- 172.21.15.143:0/2682005051 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f87c002f050 con 0x7f87d8157c70 2024-01-16T15:01:24.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.544+0000 7f87ce000640 1 --2- 172.21.15.143:0/2682005051 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f87b006b2c0 0x7f87b006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:24.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.544+0000 7f87ce000640 1 -- 172.21.15.143:0/2682005051 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f87c00aba40 con 0x7f87d8157c70 2024-01-16T15:01:24.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.545+0000 7f87d5e00640 1 --2- 172.21.15.143:0/2682005051 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f87b006b2c0 0x7f87b006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:24.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.545+0000 7f87d5e00640 1 --2- 172.21.15.143:0/2682005051 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f87b006b2c0 0x7f87b006d780 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f87c8000f60 tx=0x7f87c8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:24.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.549+0000 7f87ce000640 1 -- 172.21.15.143:0/2682005051 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f87c00ade60 con 0x7f87d8157c70 2024-01-16T15:01:24.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.738+0000 7f87dcc00640 1 -- 172.21.15.143:0/2682005051 --> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f87d81615f0 con 0x7f87b006b2c0 2024-01-16T15:01:24.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.760+0000 7f87ce000640 1 -- 172.21.15.143:0/2682005051 <== mgr.4113 v2:172.21.15.143:6832/3153492115 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287070 (secure 0 0 0) 0x7f87c8001c70 con 0x7f87b006b2c0 2024-01-16T15:01:24.743 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:01:24.745 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-16T15:01:24.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.765+0000 7f87dcc00640 1 -- 172.21.15.143:0/2682005051 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f87b006b2c0 msgr2=0x7f87b006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:24.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.765+0000 7f87dcc00640 1 --2- 172.21.15.143:0/2682005051 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f87b006b2c0 0x7f87b006d780 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f87c8000f60 tx=0x7f87c8009040 comp rx=0 tx=0).stop 2024-01-16T15:01:24.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.765+0000 7f87dcc00640 1 -- 172.21.15.143:0/2682005051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87d8157c70 msgr2=0x7f87d809bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:24.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.765+0000 7f87dcc00640 1 --2- 172.21.15.143:0/2682005051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87d8157c70 0x7f87d809bb60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f87c0000c00 tx=0x7f87c00046c0 comp rx=0 tx=0).stop 2024-01-16T15:01:24.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.765+0000 7f87dcc00640 1 -- 172.21.15.143:0/2682005051 shutdown_connections 2024-01-16T15:01:24.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.765+0000 7f87dcc00640 1 --2- 172.21.15.143:0/2682005051 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f87b006b2c0 0x7f87b006d780 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:24.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.765+0000 7f87dcc00640 1 --2- 172.21.15.143:0/2682005051 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87d815dc80 0x7f87d809c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:24.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.765+0000 7f87dcc00640 1 --2- 172.21.15.143:0/2682005051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87d8157c70 0x7f87d809bb60 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:24.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.765+0000 7f87dcc00640 1 --2- 172.21.15.143:0/2682005051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87d80cfb60 0x7f87d8151e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:24.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.765+0000 7f87dcc00640 1 -- 172.21.15.143:0/2682005051 >> 172.21.15.143:0/2682005051 conn(0x7f87d806a920 msgr2=0x7f87d815f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:24.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.766+0000 7f87dcc00640 1 -- 172.21.15.143:0/2682005051 shutdown_connections 2024-01-16T15:01:24.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:24.766+0000 7f87dcc00640 1 -- 172.21.15.143:0/2682005051 wait complete. 2024-01-16T15:01:24.764 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2024-01-16T15:01:23.677277+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220380,"kb_used_data":4180,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754340,"statfs":{"total":773094113280,"available":772868444160,"internally_reserved":0,"allocated":4280320,"data_stored":2374529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.846099"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.445114+0000","last_change":"2024-01-16T15:01:19.896749+0000","last_active":"2024-01-16T15:01:21.445114+0000","last_peered":"2024-01-16T15:01:21.445114+0000","last_clean":"2024-01-16T15:01:21.445114+0000","last_became_active":"2024-01-16T15:01:19.896443+0000","last_became_peered":"2024-01-16T15:01:19.896443+0000","last_unstale":"2024-01-16T15:01:21.445114+0000","last_undegraded":"2024-01-16T15:01:21.445114+0000","last_fullsized":"2024-01-16T15:01:21.445114+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:36:47.158317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.898297+0000","last_change":"2024-01-16T15:01:19.905287+0000","last_active":"2024-01-16T15:01:20.898297+0000","last_peered":"2024-01-16T15:01:20.898297+0000","last_clean":"2024-01-16T15:01:20.898297+0000","last_became_active":"2024-01-16T15:01:19.904981+0000","last_became_peered":"2024-01-16T15:01:19.904981+0000","last_unstale":"2024-01-16T15:01:20.898297+0000","last_undegraded":"2024-01-16T15:01:20.898297+0000","last_fullsized":"2024-01-16T15:01:20.898297+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:49:04.111811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.901813+0000","last_change":"2024-01-16T15:01:19.905331+0000","last_active":"2024-01-16T15:01:20.901813+0000","last_peered":"2024-01-16T15:01:20.901813+0000","last_clean":"2024-01-16T15:01:20.901813+0000","last_became_active":"2024-01-16T15:01:19.904652+0000","last_became_peered":"2024-01-16T15:01:19.904652+0000","last_unstale":"2024-01-16T15:01:20.901813+0000","last_undegraded":"2024-01-16T15:01:20.901813+0000","last_fullsized":"2024-01-16T15:01:20.901813+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:31:08.299586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.444269+0000","last_change":"2024-01-16T15:01:19.897610+0000","last_active":"2024-01-16T15:01:21.444269+0000","last_peered":"2024-01-16T15:01:21.444269+0000","last_clean":"2024-01-16T15:01:21.444269+0000","last_became_active":"2024-01-16T15:01:19.897301+0000","last_became_peered":"2024-01-16T15:01:19.897301+0000","last_unstale":"2024-01-16T15:01:21.444269+0000","last_undegraded":"2024-01-16T15:01:21.444269+0000","last_fullsized":"2024-01-16T15:01:21.444269+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:30:42.530766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.349263+0000","last_change":"2024-01-16T15:01:19.903993+0000","last_active":"2024-01-16T15:01:21.349263+0000","last_peered":"2024-01-16T15:01:21.349263+0000","last_clean":"2024-01-16T15:01:21.349263+0000","last_became_active":"2024-01-16T15:01:19.902007+0000","last_became_peered":"2024-01-16T15:01:19.902007+0000","last_unstale":"2024-01-16T15:01:21.349263+0000","last_undegraded":"2024-01-16T15:01:21.349263+0000","last_fullsized":"2024-01-16T15:01:21.349263+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:37:11.890430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.897688+0000","last_change":"2024-01-16T15:01:19.890431+0000","last_active":"2024-01-16T15:01:20.897688+0000","last_peered":"2024-01-16T15:01:20.897688+0000","last_clean":"2024-01-16T15:01:20.897688+0000","last_became_active":"2024-01-16T15:01:19.890135+0000","last_became_peered":"2024-01-16T15:01:19.890135+0000","last_unstale":"2024-01-16T15:01:20.897688+0000","last_undegraded":"2024-01-16T15:01:20.897688+0000","last_fullsized":"2024-01-16T15:01:20.897688+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:01:00.718830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.903192+0000","last_change":"2024-01-16T15:01:19.895013+0000","last_active":"2024-01-16T15:01:20.903192+0000","last_peered":"2024-01-16T15:01:20.903192+0000","last_clean":"2024-01-16T15:01:20.903192+0000","last_became_active":"2024-01-16T15:01:19.894728+0000","last_became_peered":"2024-01-16T15:01:19.894728+0000","last_unstale":"2024-01-16T15:01:20.903192+0000","last_undegraded":"2024-01-16T15:01:20.903192+0000","last_fullsized":"2024-01-16T15:01:20.903192+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:12:08.205887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.902266+0000","last_change":"2024-01-16T15:01:19.897961+0000","last_active":"2024-01-16T15:01:20.902266+0000","last_peered":"2024-01-16T15:01:20.902266+0000","last_clean":"2024-01-16T15:01:20.902266+0000","last_became_active":"2024-01-16T15:01:19.894706+0000","last_became_peered":"2024-01-16T15:01:19.894706+0000","last_unstale":"2024-01-16T15:01:20.902266+0000","last_undegraded":"2024-01-16T15:01:20.902266+0000","last_fullsized":"2024-01-16T15:01:20.902266+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:08:18.347887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.446418+0000","last_change":"2024-01-16T15:01:19.903915+0000","last_active":"2024-01-16T15:01:21.446418+0000","last_peered":"2024-01-16T15:01:21.446418+0000","last_clean":"2024-01-16T15:01:21.446418+0000","last_became_active":"2024-01-16T15:01:19.903168+0000","last_became_peered":"2024-01-16T15:01:19.903168+0000","last_unstale":"2024-01-16T15:01:21.446418+0000","last_undegraded":"2024-01-16T15:01:21.446418+0000","last_fullsized":"2024-01-16T15:01:21.446418+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:08:18.347887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.444606+0000","last_change":"2024-01-16T15:01:19.894997+0000","last_active":"2024-01-16T15:01:21.444606+0000","last_peered":"2024-01-16T15:01:21.444606+0000","last_clean":"2024-01-16T15:01:21.444606+0000","last_became_active":"2024-01-16T15:01:19.894687+0000","last_became_peered":"2024-01-16T15:01:19.894687+0000","last_unstale":"2024-01-16T15:01:21.444606+0000","last_undegraded":"2024-01-16T15:01:21.444606+0000","last_fullsized":"2024-01-16T15:01:21.444606+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:41:02.260826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.904329+0000","last_change":"2024-01-16T15:01:19.902052+0000","last_active":"2024-01-16T15:01:20.904329+0000","last_peered":"2024-01-16T15:01:20.904329+0000","last_clean":"2024-01-16T15:01:20.904329+0000","last_became_active":"2024-01-16T15:01:19.901164+0000","last_became_peered":"2024-01-16T15:01:19.901164+0000","last_unstale":"2024-01-16T15:01:20.904329+0000","last_undegraded":"2024-01-16T15:01:20.904329+0000","last_fullsized":"2024-01-16T15:01:20.904329+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:49:04.111811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.901399+0000","last_change":"2024-01-16T15:01:19.905501+0000","last_active":"2024-01-16T15:01:20.901399+0000","last_peered":"2024-01-16T15:01:20.901399+0000","last_clean":"2024-01-16T15:01:20.901399+0000","last_became_active":"2024-01-16T15:01:19.905206+0000","last_became_peered":"2024-01-16T15:01:19.905206+0000","last_unstale":"2024-01-16T15:01:20.901399+0000","last_undegraded":"2024-01-16T15:01:20.901399+0000","last_fullsized":"2024-01-16T15:01:20.901399+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:08:18.347887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.349553+0000","last_change":"2024-01-16T15:01:19.904347+0000","last_active":"2024-01-16T15:01:21.349553+0000","last_peered":"2024-01-16T15:01:21.349553+0000","last_clean":"2024-01-16T15:01:21.349553+0000","last_became_active":"2024-01-16T15:01:19.902780+0000","last_became_peered":"2024-01-16T15:01:19.902780+0000","last_unstale":"2024-01-16T15:01:21.349553+0000","last_undegraded":"2024-01-16T15:01:21.349553+0000","last_fullsized":"2024-01-16T15:01:21.349553+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:41:22.619386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.897827+0000","last_change":"2024-01-16T15:01:19.894720+0000","last_active":"2024-01-16T15:01:20.897827+0000","last_peered":"2024-01-16T15:01:20.897827+0000","last_clean":"2024-01-16T15:01:20.897827+0000","last_became_active":"2024-01-16T15:01:19.894462+0000","last_became_peered":"2024-01-16T15:01:19.894462+0000","last_unstale":"2024-01-16T15:01:20.897827+0000","last_undegraded":"2024-01-16T15:01:20.897827+0000","last_fullsized":"2024-01-16T15:01:20.897827+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:52:44.965933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.900901+0000","last_change":"2024-01-16T15:01:19.894716+0000","last_active":"2024-01-16T15:01:20.900901+0000","last_peered":"2024-01-16T15:01:20.900901+0000","last_clean":"2024-01-16T15:01:20.900901+0000","last_became_active":"2024-01-16T15:01:19.894466+0000","last_became_peered":"2024-01-16T15:01:19.894466+0000","last_unstale":"2024-01-16T15:01:20.900901+0000","last_undegraded":"2024-01-16T15:01:20.900901+0000","last_fullsized":"2024-01-16T15:01:20.900901+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:41:02.260826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.902967+0000","last_change":"2024-01-16T15:01:19.888176+0000","last_active":"2024-01-16T15:01:20.902967+0000","last_peered":"2024-01-16T15:01:20.902967+0000","last_clean":"2024-01-16T15:01:20.902967+0000","last_became_active":"2024-01-16T15:01:19.886502+0000","last_became_peered":"2024-01-16T15:01:19.886502+0000","last_unstale":"2024-01-16T15:01:20.902967+0000","last_undegraded":"2024-01-16T15:01:20.902967+0000","last_fullsized":"2024-01-16T15:01:20.902967+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:01:00.718830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.903736+0000","last_change":"2024-01-16T15:01:19.898338+0000","last_active":"2024-01-16T15:01:20.903736+0000","last_peered":"2024-01-16T15:01:20.903736+0000","last_clean":"2024-01-16T15:01:20.903736+0000","last_became_active":"2024-01-16T15:01:19.898050+0000","last_became_peered":"2024-01-16T15:01:19.898050+0000","last_unstale":"2024-01-16T15:01:20.903736+0000","last_undegraded":"2024-01-16T15:01:20.903736+0000","last_fullsized":"2024-01-16T15:01:20.903736+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:02:00.425436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.898245+0000","last_change":"2024-01-16T15:01:19.915366+0000","last_active":"2024-01-16T15:01:20.898245+0000","last_peered":"2024-01-16T15:01:20.898245+0000","last_clean":"2024-01-16T15:01:20.898245+0000","last_became_active":"2024-01-16T15:01:19.913042+0000","last_became_peered":"2024-01-16T15:01:19.913042+0000","last_unstale":"2024-01-16T15:01:20.898245+0000","last_undegraded":"2024-01-16T15:01:20.898245+0000","last_fullsized":"2024-01-16T15:01:20.898245+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:25:30.380602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.898042+0000","last_change":"2024-01-16T15:01:19.904308+0000","last_active":"2024-01-16T15:01:20.898042+0000","last_peered":"2024-01-16T15:01:20.898042+0000","last_clean":"2024-01-16T15:01:20.898042+0000","last_became_active":"2024-01-16T15:01:19.904053+0000","last_became_peered":"2024-01-16T15:01:19.904053+0000","last_unstale":"2024-01-16T15:01:20.898042+0000","last_undegraded":"2024-01-16T15:01:20.898042+0000","last_fullsized":"2024-01-16T15:01:20.898042+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:47:30.255903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.349522+0000","last_change":"2024-01-16T15:01:19.902503+0000","last_active":"2024-01-16T15:01:21.349522+0000","last_peered":"2024-01-16T15:01:21.349522+0000","last_clean":"2024-01-16T15:01:21.349522+0000","last_became_active":"2024-01-16T15:01:19.901569+0000","last_became_peered":"2024-01-16T15:01:19.901569+0000","last_unstale":"2024-01-16T15:01:21.349522+0000","last_undegraded":"2024-01-16T15:01:21.349522+0000","last_fullsized":"2024-01-16T15:01:21.349522+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:03:57.467477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.444750+0000","last_change":"2024-01-16T15:01:19.898103+0000","last_active":"2024-01-16T15:01:21.444750+0000","last_peered":"2024-01-16T15:01:21.444750+0000","last_clean":"2024-01-16T15:01:21.444750+0000","last_became_active":"2024-01-16T15:01:19.897858+0000","last_became_peered":"2024-01-16T15:01:19.897858+0000","last_unstale":"2024-01-16T15:01:21.444750+0000","last_undegraded":"2024-01-16T15:01:21.444750+0000","last_fullsized":"2024-01-16T15:01:21.444750+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:30:42.530766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.444615+0000","last_change":"2024-01-16T15:01:19.894844+0000","last_active":"2024-01-16T15:01:21.444615+0000","last_peered":"2024-01-16T15:01:21.444615+0000","last_clean":"2024-01-16T15:01:21.444615+0000","last_became_active":"2024-01-16T15:01:19.893574+0000","last_became_peered":"2024-01-16T15:01:19.893574+0000","last_unstale":"2024-01-16T15:01:21.444615+0000","last_undegraded":"2024-01-16T15:01:21.444615+0000","last_fullsized":"2024-01-16T15:01:21.444615+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:46:28.675655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.442171+0000","last_change":"2024-01-16T15:01:19.877123+0000","last_active":"2024-01-16T15:01:21.442171+0000","last_peered":"2024-01-16T15:01:21.442171+0000","last_clean":"2024-01-16T15:01:21.442171+0000","last_became_active":"2024-01-16T15:01:19.876406+0000","last_became_peered":"2024-01-16T15:01:19.876406+0000","last_unstale":"2024-01-16T15:01:21.442171+0000","last_undegraded":"2024-01-16T15:01:21.442171+0000","last_fullsized":"2024-01-16T15:01:21.442171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:14:12.867834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.900932+0000","last_change":"2024-01-16T15:01:19.902197+0000","last_active":"2024-01-16T15:01:20.900932+0000","last_peered":"2024-01-16T15:01:20.900932+0000","last_clean":"2024-01-16T15:01:20.900932+0000","last_became_active":"2024-01-16T15:01:19.901878+0000","last_became_peered":"2024-01-16T15:01:19.901878+0000","last_unstale":"2024-01-16T15:01:20.900932+0000","last_undegraded":"2024-01-16T15:01:20.900932+0000","last_fullsized":"2024-01-16T15:01:20.900932+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:36:30.879279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.444785+0000","last_change":"2024-01-16T15:01:19.898875+0000","last_active":"2024-01-16T15:01:21.444785+0000","last_peered":"2024-01-16T15:01:21.444785+0000","last_clean":"2024-01-16T15:01:21.444785+0000","last_became_active":"2024-01-16T15:01:19.898575+0000","last_became_peered":"2024-01-16T15:01:19.898575+0000","last_unstale":"2024-01-16T15:01:21.444785+0000","last_undegraded":"2024-01-16T15:01:21.444785+0000","last_fullsized":"2024-01-16T15:01:21.444785+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:25:29.059223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.898387+0000","last_change":"2024-01-16T15:01:19.893928+0000","last_active":"2024-01-16T15:01:20.898387+0000","last_peered":"2024-01-16T15:01:20.898387+0000","last_clean":"2024-01-16T15:01:20.898387+0000","last_became_active":"2024-01-16T15:01:19.893678+0000","last_became_peered":"2024-01-16T15:01:19.893678+0000","last_unstale":"2024-01-16T15:01:20.898387+0000","last_undegraded":"2024-01-16T15:01:20.898387+0000","last_fullsized":"2024-01-16T15:01:20.898387+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:01:23.441562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.445123+0000","last_change":"2024-01-16T15:01:19.898480+0000","last_active":"2024-01-16T15:01:21.445123+0000","last_peered":"2024-01-16T15:01:21.445123+0000","last_clean":"2024-01-16T15:01:21.445123+0000","last_became_active":"2024-01-16T15:01:19.898161+0000","last_became_peered":"2024-01-16T15:01:19.898161+0000","last_unstale":"2024-01-16T15:01:21.445123+0000","last_undegraded":"2024-01-16T15:01:21.445123+0000","last_fullsized":"2024-01-16T15:01:21.445123+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:08:18.347887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.901900+0000","last_change":"2024-01-16T15:01:19.901348+0000","last_active":"2024-01-16T15:01:20.901900+0000","last_peered":"2024-01-16T15:01:20.901900+0000","last_clean":"2024-01-16T15:01:20.901900+0000","last_became_active":"2024-01-16T15:01:19.900965+0000","last_became_peered":"2024-01-16T15:01:19.900965+0000","last_unstale":"2024-01-16T15:01:20.901900+0000","last_undegraded":"2024-01-16T15:01:20.901900+0000","last_fullsized":"2024-01-16T15:01:20.901900+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:01:00.718830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.445039+0000","last_change":"2024-01-16T15:01:19.893658+0000","last_active":"2024-01-16T15:01:21.445039+0000","last_peered":"2024-01-16T15:01:21.445039+0000","last_clean":"2024-01-16T15:01:21.445039+0000","last_became_active":"2024-01-16T15:01:19.893342+0000","last_became_peered":"2024-01-16T15:01:19.893342+0000","last_unstale":"2024-01-16T15:01:21.445039+0000","last_undegraded":"2024-01-16T15:01:21.445039+0000","last_fullsized":"2024-01-16T15:01:21.445039+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:41:02.260826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.903326+0000","last_change":"2024-01-16T15:01:19.900634+0000","last_active":"2024-01-16T15:01:20.903326+0000","last_peered":"2024-01-16T15:01:20.903326+0000","last_clean":"2024-01-16T15:01:20.903326+0000","last_became_active":"2024-01-16T15:01:19.900337+0000","last_became_peered":"2024-01-16T15:01:19.900337+0000","last_unstale":"2024-01-16T15:01:20.903326+0000","last_undegraded":"2024-01-16T15:01:20.903326+0000","last_fullsized":"2024-01-16T15:01:20.903326+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:47:30.255903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.443980+0000","last_change":"2024-01-16T15:01:19.897073+0000","last_active":"2024-01-16T15:01:21.443980+0000","last_peered":"2024-01-16T15:01:21.443980+0000","last_clean":"2024-01-16T15:01:21.443980+0000","last_became_active":"2024-01-16T15:01:19.896788+0000","last_became_peered":"2024-01-16T15:01:19.896788+0000","last_unstale":"2024-01-16T15:01:21.443980+0000","last_undegraded":"2024-01-16T15:01:21.443980+0000","last_fullsized":"2024-01-16T15:01:21.443980+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:22:03.634466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.902293+0000","last_change":"2024-01-16T15:01:19.890441+0000","last_active":"2024-01-16T15:01:20.902293+0000","last_peered":"2024-01-16T15:01:20.902293+0000","last_clean":"2024-01-16T15:01:20.902293+0000","last_became_active":"2024-01-16T15:01:19.890076+0000","last_became_peered":"2024-01-16T15:01:19.890076+0000","last_unstale":"2024-01-16T15:01:20.902293+0000","last_undegraded":"2024-01-16T15:01:20.902293+0000","last_fullsized":"2024-01-16T15:01:20.902293+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:15:15.239596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.442964+0000","last_change":"2024-01-16T15:01:19.889683+0000","last_active":"2024-01-16T15:01:21.442964+0000","last_peered":"2024-01-16T15:01:21.442964+0000","last_clean":"2024-01-16T15:01:21.442964+0000","last_became_active":"2024-01-16T15:01:19.881670+0000","last_became_peered":"2024-01-16T15:01:19.881670+0000","last_unstale":"2024-01-16T15:01:21.442964+0000","last_undegraded":"2024-01-16T15:01:21.442964+0000","last_fullsized":"2024-01-16T15:01:21.442964+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:46:28.675655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.445661+0000","last_change":"2024-01-16T15:01:18.877659+0000","last_active":"2024-01-16T15:01:21.445661+0000","last_peered":"2024-01-16T15:01:21.445661+0000","last_clean":"2024-01-16T15:01:21.445661+0000","last_became_active":"2024-01-16T15:01:18.877324+0000","last_became_peered":"2024-01-16T15:01:18.877324+0000","last_unstale":"2024-01-16T15:01:21.445661+0000","last_undegraded":"2024-01-16T15:01:21.445661+0000","last_fullsized":"2024-01-16T15:01:21.445661+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:11:58.134390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.445482+0000","last_change":"2024-01-16T15:01:19.892754+0000","last_active":"2024-01-16T15:01:21.445482+0000","last_peered":"2024-01-16T15:01:21.445482+0000","last_clean":"2024-01-16T15:01:21.445482+0000","last_became_active":"2024-01-16T15:01:19.892461+0000","last_became_peered":"2024-01-16T15:01:19.892461+0000","last_unstale":"2024-01-16T15:01:21.445482+0000","last_undegraded":"2024-01-16T15:01:21.445482+0000","last_fullsized":"2024-01-16T15:01:21.445482+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:35:31.837292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.442834+0000","last_change":"2024-01-16T15:01:18.874834+0000","last_active":"2024-01-16T15:01:21.442834+0000","last_peered":"2024-01-16T15:01:21.442834+0000","last_clean":"2024-01-16T15:01:21.442834+0000","last_became_active":"2024-01-16T15:01:18.874602+0000","last_became_peered":"2024-01-16T15:01:18.874602+0000","last_unstale":"2024-01-16T15:01:21.442834+0000","last_undegraded":"2024-01-16T15:01:21.442834+0000","last_fullsized":"2024-01-16T15:01:21.442834+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:11:58.134390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.445916+0000","last_change":"2024-01-16T15:01:19.895059+0000","last_active":"2024-01-16T15:01:21.445916+0000","last_peered":"2024-01-16T15:01:21.445916+0000","last_clean":"2024-01-16T15:01:21.445916+0000","last_became_active":"2024-01-16T15:01:19.894743+0000","last_became_peered":"2024-01-16T15:01:19.894743+0000","last_unstale":"2024-01-16T15:01:21.445916+0000","last_undegraded":"2024-01-16T15:01:21.445916+0000","last_fullsized":"2024-01-16T15:01:21.445916+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:36:30.879279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.898017+0000","last_change":"2024-01-16T15:01:18.857052+0000","last_active":"2024-01-16T15:01:20.898017+0000","last_peered":"2024-01-16T15:01:20.898017+0000","last_clean":"2024-01-16T15:01:20.898017+0000","last_became_active":"2024-01-16T15:01:18.856769+0000","last_became_peered":"2024-01-16T15:01:18.856769+0000","last_unstale":"2024-01-16T15:01:20.898017+0000","last_undegraded":"2024-01-16T15:01:20.898017+0000","last_fullsized":"2024-01-16T15:01:20.898017+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:15:14.228212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.445168+0000","last_change":"2024-01-16T15:01:19.897898+0000","last_active":"2024-01-16T15:01:21.445168+0000","last_peered":"2024-01-16T15:01:21.445168+0000","last_clean":"2024-01-16T15:01:21.445168+0000","last_became_active":"2024-01-16T15:01:19.894918+0000","last_became_peered":"2024-01-16T15:01:19.894918+0000","last_unstale":"2024-01-16T15:01:21.445168+0000","last_undegraded":"2024-01-16T15:01:21.445168+0000","last_fullsized":"2024-01-16T15:01:21.445168+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:15:15.239596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.349565+0000","last_change":"2024-01-16T15:01:18.863397+0000","last_active":"2024-01-16T15:01:21.349565+0000","last_peered":"2024-01-16T15:01:21.349565+0000","last_clean":"2024-01-16T15:01:21.349565+0000","last_became_active":"2024-01-16T15:01:18.863134+0000","last_became_peered":"2024-01-16T15:01:18.863134+0000","last_unstale":"2024-01-16T15:01:21.349565+0000","last_undegraded":"2024-01-16T15:01:21.349565+0000","last_fullsized":"2024-01-16T15:01:21.349565+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:32:47.434130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.903501+0000","last_change":"2024-01-16T15:01:19.882788+0000","last_active":"2024-01-16T15:01:20.903501+0000","last_peered":"2024-01-16T15:01:20.903501+0000","last_clean":"2024-01-16T15:01:20.903501+0000","last_became_active":"2024-01-16T15:01:19.881913+0000","last_became_peered":"2024-01-16T15:01:19.881913+0000","last_unstale":"2024-01-16T15:01:20.903501+0000","last_undegraded":"2024-01-16T15:01:20.903501+0000","last_fullsized":"2024-01-16T15:01:20.903501+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:25:30.380602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.445009+0000","last_change":"2024-01-16T15:01:18.865837+0000","last_active":"2024-01-16T15:01:21.445009+0000","last_peered":"2024-01-16T15:01:21.445009+0000","last_clean":"2024-01-16T15:01:21.445009+0000","last_became_active":"2024-01-16T15:01:18.863538+0000","last_became_peered":"2024-01-16T15:01:18.863538+0000","last_unstale":"2024-01-16T15:01:21.445009+0000","last_undegraded":"2024-01-16T15:01:21.445009+0000","last_fullsized":"2024-01-16T15:01:21.445009+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:03:48.869753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.901091+0000","last_change":"2024-01-16T15:01:19.905185+0000","last_active":"2024-01-16T15:01:20.901091+0000","last_peered":"2024-01-16T15:01:20.901091+0000","last_clean":"2024-01-16T15:01:20.901091+0000","last_became_active":"2024-01-16T15:01:19.904821+0000","last_became_peered":"2024-01-16T15:01:19.904821+0000","last_unstale":"2024-01-16T15:01:20.901091+0000","last_undegraded":"2024-01-16T15:01:20.901091+0000","last_fullsized":"2024-01-16T15:01:20.901091+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:25:29.059223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.444699+0000","last_change":"2024-01-16T15:01:18.868539+0000","last_active":"2024-01-16T15:01:21.444699+0000","last_peered":"2024-01-16T15:01:21.444699+0000","last_clean":"2024-01-16T15:01:21.444699+0000","last_became_active":"2024-01-16T15:01:18.868205+0000","last_became_peered":"2024-01-16T15:01:18.868205+0000","last_unstale":"2024-01-16T15:01:21.444699+0000","last_undegraded":"2024-01-16T15:01:21.444699+0000","last_fullsized":"2024-01-16T15:01:21.444699+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:14:11.856450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.444082+0000","last_change":"2024-01-16T15:01:19.888191+0000","last_active":"2024-01-16T15:01:21.444082+0000","last_peered":"2024-01-16T15:01:21.444082+0000","last_clean":"2024-01-16T15:01:21.444082+0000","last_became_active":"2024-01-16T15:01:19.887906+0000","last_became_peered":"2024-01-16T15:01:19.887906+0000","last_unstale":"2024-01-16T15:01:21.444082+0000","last_undegraded":"2024-01-16T15:01:21.444082+0000","last_fullsized":"2024-01-16T15:01:21.444082+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:35:31.837292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.897495+0000","last_change":"2024-01-16T15:01:18.856602+0000","last_active":"2024-01-16T15:01:20.897495+0000","last_peered":"2024-01-16T15:01:20.897495+0000","last_clean":"2024-01-16T15:01:20.897495+0000","last_became_active":"2024-01-16T15:01:18.855724+0000","last_became_peered":"2024-01-16T15:01:18.855724+0000","last_unstale":"2024-01-16T15:01:20.897495+0000","last_undegraded":"2024-01-16T15:01:20.897495+0000","last_fullsized":"2024-01-16T15:01:20.897495+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:46:27.664271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.443381+0000","last_change":"2024-01-16T15:01:19.895075+0000","last_active":"2024-01-16T15:01:21.443381+0000","last_peered":"2024-01-16T15:01:21.443381+0000","last_clean":"2024-01-16T15:01:21.443381+0000","last_became_active":"2024-01-16T15:01:19.894819+0000","last_became_peered":"2024-01-16T15:01:19.894819+0000","last_unstale":"2024-01-16T15:01:21.443381+0000","last_undegraded":"2024-01-16T15:01:21.443381+0000","last_fullsized":"2024-01-16T15:01:21.443381+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:35:31.837292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.444283+0000","last_change":"2024-01-16T15:01:18.870847+0000","last_active":"2024-01-16T15:01:21.444283+0000","last_peered":"2024-01-16T15:01:21.444283+0000","last_clean":"2024-01-16T15:01:21.444283+0000","last_became_active":"2024-01-16T15:01:18.870559+0000","last_became_peered":"2024-01-16T15:01:18.870559+0000","last_unstale":"2024-01-16T15:01:21.444283+0000","last_undegraded":"2024-01-16T15:01:21.444283+0000","last_fullsized":"2024-01-16T15:01:21.444283+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:37:10.879045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.902097+0000","last_change":"2024-01-16T15:01:19.905944+0000","last_active":"2024-01-16T15:01:20.902097+0000","last_peered":"2024-01-16T15:01:20.902097+0000","last_clean":"2024-01-16T15:01:20.902097+0000","last_became_active":"2024-01-16T15:01:19.905659+0000","last_became_peered":"2024-01-16T15:01:19.905659+0000","last_unstale":"2024-01-16T15:01:20.902097+0000","last_undegraded":"2024-01-16T15:01:20.902097+0000","last_fullsized":"2024-01-16T15:01:20.902097+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:12:08.205887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.898329+0000","last_change":"2024-01-16T15:01:18.870502+0000","last_active":"2024-01-16T15:01:20.898329+0000","last_peered":"2024-01-16T15:01:20.898329+0000","last_clean":"2024-01-16T15:01:20.898329+0000","last_became_active":"2024-01-16T15:01:18.870044+0000","last_became_peered":"2024-01-16T15:01:18.870044+0000","last_unstale":"2024-01-16T15:01:20.898329+0000","last_undegraded":"2024-01-16T15:01:20.898329+0000","last_fullsized":"2024-01-16T15:01:20.898329+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:08:17.336503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.898272+0000","last_change":"2024-01-16T15:01:19.895553+0000","last_active":"2024-01-16T15:01:20.898272+0000","last_peered":"2024-01-16T15:01:20.898272+0000","last_clean":"2024-01-16T15:01:20.898272+0000","last_became_active":"2024-01-16T15:01:19.894475+0000","last_became_peered":"2024-01-16T15:01:19.894475+0000","last_unstale":"2024-01-16T15:01:20.898272+0000","last_undegraded":"2024-01-16T15:01:20.898272+0000","last_fullsized":"2024-01-16T15:01:20.898272+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:02:00.425436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.443073+0000","last_change":"2024-01-16T15:01:18.863763+0000","last_active":"2024-01-16T15:01:21.443073+0000","last_peered":"2024-01-16T15:01:21.443073+0000","last_clean":"2024-01-16T15:01:21.443073+0000","last_became_active":"2024-01-16T15:01:18.863388+0000","last_became_peered":"2024-01-16T15:01:18.863388+0000","last_unstale":"2024-01-16T15:01:21.443073+0000","last_undegraded":"2024-01-16T15:01:21.443073+0000","last_fullsized":"2024-01-16T15:01:21.443073+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:57:03.106778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.898124+0000","last_change":"2024-01-16T15:01:19.892719+0000","last_active":"2024-01-16T15:01:20.898124+0000","last_peered":"2024-01-16T15:01:20.898124+0000","last_clean":"2024-01-16T15:01:20.898124+0000","last_became_active":"2024-01-16T15:01:19.892442+0000","last_became_peered":"2024-01-16T15:01:19.892442+0000","last_unstale":"2024-01-16T15:01:20.898124+0000","last_undegraded":"2024-01-16T15:01:20.898124+0000","last_fullsized":"2024-01-16T15:01:20.898124+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:43:37.821973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.901985+0000","last_change":"2024-01-16T15:01:18.875621+0000","last_active":"2024-01-16T15:01:20.901985+0000","last_peered":"2024-01-16T15:01:20.901985+0000","last_clean":"2024-01-16T15:01:20.901985+0000","last_became_active":"2024-01-16T15:01:18.874654+0000","last_became_peered":"2024-01-16T15:01:18.874654+0000","last_unstale":"2024-01-16T15:01:20.901985+0000","last_undegraded":"2024-01-16T15:01:20.901985+0000","last_fullsized":"2024-01-16T15:01:20.901985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:14:11.856450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.901725+0000","last_change":"2024-01-16T15:01:19.905409+0000","last_active":"2024-01-16T15:01:20.901725+0000","last_peered":"2024-01-16T15:01:20.901725+0000","last_clean":"2024-01-16T15:01:20.901725+0000","last_became_active":"2024-01-16T15:01:19.905039+0000","last_became_peered":"2024-01-16T15:01:19.905039+0000","last_unstale":"2024-01-16T15:01:20.901725+0000","last_undegraded":"2024-01-16T15:01:20.901725+0000","last_fullsized":"2024-01-16T15:01:20.901725+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:15:15.239596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.898300+0000","last_change":"2024-01-16T15:01:18.862762+0000","last_active":"2024-01-16T15:01:20.898300+0000","last_peered":"2024-01-16T15:01:20.898300+0000","last_clean":"2024-01-16T15:01:20.898300+0000","last_became_active":"2024-01-16T15:01:18.862467+0000","last_became_peered":"2024-01-16T15:01:18.862467+0000","last_unstale":"2024-01-16T15:01:20.898300+0000","last_undegraded":"2024-01-16T15:01:20.898300+0000","last_fullsized":"2024-01-16T15:01:20.898300+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:25:28.047839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.443917+0000","last_change":"2024-01-16T15:01:19.893917+0000","last_active":"2024-01-16T15:01:21.443917+0000","last_peered":"2024-01-16T15:01:21.443917+0000","last_clean":"2024-01-16T15:01:21.443917+0000","last_became_active":"2024-01-16T15:01:19.893635+0000","last_became_peered":"2024-01-16T15:01:19.893635+0000","last_unstale":"2024-01-16T15:01:21.443917+0000","last_undegraded":"2024-01-16T15:01:21.443917+0000","last_fullsized":"2024-01-16T15:01:21.443917+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:46:28.675655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.445253+0000","last_change":"2024-01-16T15:01:18.872168+0000","last_active":"2024-01-16T15:01:21.445253+0000","last_peered":"2024-01-16T15:01:21.445253+0000","last_clean":"2024-01-16T15:01:21.445253+0000","last_became_active":"2024-01-16T15:01:18.871737+0000","last_became_peered":"2024-01-16T15:01:18.871737+0000","last_unstale":"2024-01-16T15:01:21.445253+0000","last_undegraded":"2024-01-16T15:01:21.445253+0000","last_fullsized":"2024-01-16T15:01:21.445253+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:57:03.106778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.897763+0000","last_change":"2024-01-16T15:01:19.906704+0000","last_active":"2024-01-16T15:01:20.897763+0000","last_peered":"2024-01-16T15:01:20.897763+0000","last_clean":"2024-01-16T15:01:20.897763+0000","last_became_active":"2024-01-16T15:01:19.906296+0000","last_became_peered":"2024-01-16T15:01:19.906296+0000","last_unstale":"2024-01-16T15:01:20.897763+0000","last_undegraded":"2024-01-16T15:01:20.897763+0000","last_fullsized":"2024-01-16T15:01:20.897763+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:12:08.205887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.443884+0000","last_change":"2024-01-16T15:01:18.871031+0000","last_active":"2024-01-16T15:01:21.443884+0000","last_peered":"2024-01-16T15:01:21.443884+0000","last_clean":"2024-01-16T15:01:21.443884+0000","last_became_active":"2024-01-16T15:01:18.870735+0000","last_became_peered":"2024-01-16T15:01:18.870735+0000","last_unstale":"2024-01-16T15:01:21.443884+0000","last_undegraded":"2024-01-16T15:01:21.443884+0000","last_fullsized":"2024-01-16T15:01:21.443884+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:11:58.134390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.902725+0000","last_change":"2024-01-16T15:01:19.884489+0000","last_active":"2024-01-16T15:01:20.902725+0000","last_peered":"2024-01-16T15:01:20.902725+0000","last_clean":"2024-01-16T15:01:20.902725+0000","last_became_active":"2024-01-16T15:01:19.884084+0000","last_became_peered":"2024-01-16T15:01:19.884084+0000","last_unstale":"2024-01-16T15:01:20.902725+0000","last_undegraded":"2024-01-16T15:01:20.902725+0000","last_fullsized":"2024-01-16T15:01:20.902725+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:31:08.299586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.897487+0000","last_change":"2024-01-16T15:01:18.865261+0000","last_active":"2024-01-16T15:01:20.897487+0000","last_peered":"2024-01-16T15:01:20.897487+0000","last_clean":"2024-01-16T15:01:20.897487+0000","last_became_active":"2024-01-16T15:01:18.864967+0000","last_became_peered":"2024-01-16T15:01:18.864967+0000","last_unstale":"2024-01-16T15:01:20.897487+0000","last_undegraded":"2024-01-16T15:01:20.897487+0000","last_fullsized":"2024-01-16T15:01:20.897487+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:30:41.519382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.444324+0000","last_change":"2024-01-16T15:01:19.895106+0000","last_active":"2024-01-16T15:01:21.444324+0000","last_peered":"2024-01-16T15:01:21.444324+0000","last_clean":"2024-01-16T15:01:21.444324+0000","last_became_active":"2024-01-16T15:01:19.892683+0000","last_became_peered":"2024-01-16T15:01:19.892683+0000","last_unstale":"2024-01-16T15:01:21.444324+0000","last_undegraded":"2024-01-16T15:01:21.444324+0000","last_fullsized":"2024-01-16T15:01:21.444324+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:41:22.704472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":20,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.897376+0000","last_change":"2024-01-16T15:01:18.863963+0000","last_active":"2024-01-16T15:01:20.897376+0000","last_peered":"2024-01-16T15:01:20.897376+0000","last_clean":"2024-01-16T15:01:20.897376+0000","last_became_active":"2024-01-16T15:01:18.861667+0000","last_became_peered":"2024-01-16T15:01:18.861667+0000","last_unstale":"2024-01-16T15:01:20.897376+0000","last_undegraded":"2024-01-16T15:01:20.897376+0000","last_fullsized":"2024-01-16T15:01:20.897376+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:14:11.856450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.446022+0000","last_change":"2024-01-16T15:01:19.904229+0000","last_active":"2024-01-16T15:01:21.446022+0000","last_peered":"2024-01-16T15:01:21.446022+0000","last_clean":"2024-01-16T15:01:21.446022+0000","last_became_active":"2024-01-16T15:01:19.903865+0000","last_became_peered":"2024-01-16T15:01:19.903865+0000","last_unstale":"2024-01-16T15:01:21.446022+0000","last_undegraded":"2024-01-16T15:01:21.446022+0000","last_fullsized":"2024-01-16T15:01:21.446022+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:25:29.059223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.444938+0000","last_change":"2024-01-16T15:01:18.860430+0000","last_active":"2024-01-16T15:01:21.444938+0000","last_peered":"2024-01-16T15:01:21.444938+0000","last_clean":"2024-01-16T15:01:21.444938+0000","last_became_active":"2024-01-16T15:01:18.860139+0000","last_became_peered":"2024-01-16T15:01:18.860139+0000","last_unstale":"2024-01-16T15:01:21.444938+0000","last_undegraded":"2024-01-16T15:01:21.444938+0000","last_fullsized":"2024-01-16T15:01:21.444938+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:41:21.693088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.902497+0000","last_change":"2024-01-16T15:01:19.901362+0000","last_active":"2024-01-16T15:01:20.902497+0000","last_peered":"2024-01-16T15:01:20.902497+0000","last_clean":"2024-01-16T15:01:20.902497+0000","last_became_active":"2024-01-16T15:01:19.900896+0000","last_became_peered":"2024-01-16T15:01:19.900896+0000","last_unstale":"2024-01-16T15:01:20.902497+0000","last_undegraded":"2024-01-16T15:01:20.902497+0000","last_fullsized":"2024-01-16T15:01:20.902497+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:47:30.255903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.445908+0000","last_change":"2024-01-16T15:01:18.872423+0000","last_active":"2024-01-16T15:01:21.445908+0000","last_peered":"2024-01-16T15:01:21.445908+0000","last_clean":"2024-01-16T15:01:21.445908+0000","last_became_active":"2024-01-16T15:01:18.872040+0000","last_became_peered":"2024-01-16T15:01:18.872040+0000","last_unstale":"2024-01-16T15:01:21.445908+0000","last_undegraded":"2024-01-16T15:01:21.445908+0000","last_fullsized":"2024-01-16T15:01:21.445908+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:11:58.134390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.444642+0000","last_change":"2024-01-16T15:01:19.902396+0000","last_active":"2024-01-16T15:01:21.444642+0000","last_peered":"2024-01-16T15:01:21.444642+0000","last_clean":"2024-01-16T15:01:21.444642+0000","last_became_active":"2024-01-16T15:01:19.902052+0000","last_became_peered":"2024-01-16T15:01:19.902052+0000","last_unstale":"2024-01-16T15:01:21.444642+0000","last_undegraded":"2024-01-16T15:01:21.444642+0000","last_fullsized":"2024-01-16T15:01:21.444642+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:36:30.879279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.446228+0000","last_change":"2024-01-16T15:01:18.862214+0000","last_active":"2024-01-16T15:01:21.446228+0000","last_peered":"2024-01-16T15:01:21.446228+0000","last_clean":"2024-01-16T15:01:21.446228+0000","last_became_active":"2024-01-16T15:01:18.861928+0000","last_became_peered":"2024-01-16T15:01:18.861928+0000","last_unstale":"2024-01-16T15:01:21.446228+0000","last_undegraded":"2024-01-16T15:01:21.446228+0000","last_fullsized":"2024-01-16T15:01:21.446228+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:03:56.456093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.349806+0000","last_change":"2024-01-16T15:01:19.894249+0000","last_active":"2024-01-16T15:01:21.349806+0000","last_peered":"2024-01-16T15:01:21.349806+0000","last_clean":"2024-01-16T15:01:21.349806+0000","last_became_active":"2024-01-16T15:01:19.893879+0000","last_became_peered":"2024-01-16T15:01:19.893879+0000","last_unstale":"2024-01-16T15:01:21.349806+0000","last_undegraded":"2024-01-16T15:01:21.349806+0000","last_fullsized":"2024-01-16T15:01:21.349806+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:12:29.462524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.446140+0000","last_change":"2024-01-16T15:01:18.868277+0000","last_active":"2024-01-16T15:01:21.446140+0000","last_peered":"2024-01-16T15:01:21.446140+0000","last_clean":"2024-01-16T15:01:21.446140+0000","last_became_active":"2024-01-16T15:01:18.867949+0000","last_became_peered":"2024-01-16T15:01:18.867949+0000","last_unstale":"2024-01-16T15:01:21.446140+0000","last_undegraded":"2024-01-16T15:01:21.446140+0000","last_fullsized":"2024-01-16T15:01:21.446140+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:41:21.693088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.349192+0000","last_change":"2024-01-16T15:01:19.886196+0000","last_active":"2024-01-16T15:01:21.349192+0000","last_peered":"2024-01-16T15:01:21.349192+0000","last_clean":"2024-01-16T15:01:21.349192+0000","last_became_active":"2024-01-16T15:01:19.885784+0000","last_became_peered":"2024-01-16T15:01:19.885784+0000","last_unstale":"2024-01-16T15:01:21.349192+0000","last_undegraded":"2024-01-16T15:01:21.349192+0000","last_fullsized":"2024-01-16T15:01:21.349192+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:06:45.119287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.442895+0000","last_change":"2024-01-16T15:01:18.864458+0000","last_active":"2024-01-16T15:01:21.442895+0000","last_peered":"2024-01-16T15:01:21.442895+0000","last_clean":"2024-01-16T15:01:21.442895+0000","last_became_active":"2024-01-16T15:01:18.864143+0000","last_became_peered":"2024-01-16T15:01:18.864143+0000","last_unstale":"2024-01-16T15:01:21.442895+0000","last_undegraded":"2024-01-16T15:01:21.442895+0000","last_fullsized":"2024-01-16T15:01:21.442895+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:03:56.456093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.349731+0000","last_change":"2024-01-16T15:01:19.899007+0000","last_active":"2024-01-16T15:01:21.349731+0000","last_peered":"2024-01-16T15:01:21.349731+0000","last_clean":"2024-01-16T15:01:21.349731+0000","last_became_active":"2024-01-16T15:01:19.898356+0000","last_became_peered":"2024-01-16T15:01:19.898356+0000","last_unstale":"2024-01-16T15:01:21.349731+0000","last_undegraded":"2024-01-16T15:01:21.349731+0000","last_fullsized":"2024-01-16T15:01:21.349731+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:09:03.014818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.445307+0000","last_change":"2024-01-16T15:01:18.875155+0000","last_active":"2024-01-16T15:01:21.445307+0000","last_peered":"2024-01-16T15:01:21.445307+0000","last_clean":"2024-01-16T15:01:21.445307+0000","last_became_active":"2024-01-16T15:01:18.874825+0000","last_became_peered":"2024-01-16T15:01:18.874825+0000","last_unstale":"2024-01-16T15:01:21.445307+0000","last_undegraded":"2024-01-16T15:01:21.445307+0000","last_fullsized":"2024-01-16T15:01:21.445307+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:57:03.106778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.349305+0000","last_change":"2024-01-16T15:01:19.897044+0000","last_active":"2024-01-16T15:01:21.349305+0000","last_peered":"2024-01-16T15:01:21.349305+0000","last_clean":"2024-01-16T15:01:21.349305+0000","last_became_active":"2024-01-16T15:01:19.896691+0000","last_became_peered":"2024-01-16T15:01:19.896691+0000","last_unstale":"2024-01-16T15:01:21.349305+0000","last_undegraded":"2024-01-16T15:01:21.349305+0000","last_fullsized":"2024-01-16T15:01:21.349305+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:22:11.277546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.444099+0000","last_change":"2024-01-16T15:01:18.862881+0000","last_active":"2024-01-16T15:01:21.444099+0000","last_peered":"2024-01-16T15:01:21.444099+0000","last_clean":"2024-01-16T15:01:21.444099+0000","last_became_active":"2024-01-16T15:01:18.862433+0000","last_became_peered":"2024-01-16T15:01:18.862433+0000","last_unstale":"2024-01-16T15:01:21.444099+0000","last_undegraded":"2024-01-16T15:01:21.444099+0000","last_fullsized":"2024-01-16T15:01:21.444099+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:54:04.021554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.903202+0000","last_change":"2024-01-16T15:01:19.891593+0000","last_active":"2024-01-16T15:01:20.903202+0000","last_peered":"2024-01-16T15:01:20.903202+0000","last_clean":"2024-01-16T15:01:20.903202+0000","last_became_active":"2024-01-16T15:01:19.890549+0000","last_became_peered":"2024-01-16T15:01:19.890549+0000","last_unstale":"2024-01-16T15:01:20.903202+0000","last_undegraded":"2024-01-16T15:01:20.903202+0000","last_fullsized":"2024-01-16T15:01:20.903202+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:52:44.965933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.900696+0000","last_change":"2024-01-16T15:01:18.876724+0000","last_active":"2024-01-16T15:01:20.900696+0000","last_peered":"2024-01-16T15:01:20.900696+0000","last_clean":"2024-01-16T15:01:20.900696+0000","last_became_active":"2024-01-16T15:01:18.875910+0000","last_became_peered":"2024-01-16T15:01:18.875910+0000","last_unstale":"2024-01-16T15:01:20.900696+0000","last_undegraded":"2024-01-16T15:01:20.900696+0000","last_fullsized":"2024-01-16T15:01:20.900696+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:12:28.451140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.349664+0000","last_change":"2024-01-16T15:01:19.904235+0000","last_active":"2024-01-16T15:01:21.349664+0000","last_peered":"2024-01-16T15:01:21.349664+0000","last_clean":"2024-01-16T15:01:21.349664+0000","last_became_active":"2024-01-16T15:01:19.903591+0000","last_became_peered":"2024-01-16T15:01:19.903591+0000","last_unstale":"2024-01-16T15:01:21.349664+0000","last_undegraded":"2024-01-16T15:01:21.349664+0000","last_fullsized":"2024-01-16T15:01:21.349664+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:54:05.032938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.445023+0000","last_change":"2024-01-16T15:01:18.876461+0000","last_active":"2024-01-16T15:01:21.445023+0000","last_peered":"2024-01-16T15:01:21.445023+0000","last_clean":"2024-01-16T15:01:21.445023+0000","last_became_active":"2024-01-16T15:01:18.876220+0000","last_became_peered":"2024-01-16T15:01:18.876220+0000","last_unstale":"2024-01-16T15:01:21.445023+0000","last_undegraded":"2024-01-16T15:01:21.445023+0000","last_fullsized":"2024-01-16T15:01:21.445023+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:14:11.856450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.902491+0000","last_change":"2024-01-16T15:01:19.904111+0000","last_active":"2024-01-16T15:01:20.902491+0000","last_peered":"2024-01-16T15:01:20.902491+0000","last_clean":"2024-01-16T15:01:20.902491+0000","last_became_active":"2024-01-16T15:01:19.903825+0000","last_became_peered":"2024-01-16T15:01:19.903825+0000","last_unstale":"2024-01-16T15:01:20.902491+0000","last_undegraded":"2024-01-16T15:01:20.902491+0000","last_fullsized":"2024-01-16T15:01:20.902491+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:52:44.965933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.443185+0000","last_change":"2024-01-16T15:01:18.865194+0000","last_active":"2024-01-16T15:01:21.443185+0000","last_peered":"2024-01-16T15:01:21.443185+0000","last_clean":"2024-01-16T15:01:21.443185+0000","last_became_active":"2024-01-16T15:01:18.864944+0000","last_became_peered":"2024-01-16T15:01:18.864944+0000","last_unstale":"2024-01-16T15:01:21.443185+0000","last_undegraded":"2024-01-16T15:01:21.443185+0000","last_fullsized":"2024-01-16T15:01:21.443185+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:22:02.623082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.442573+0000","last_change":"2024-01-16T15:01:19.910243+0000","last_active":"2024-01-16T15:01:21.442573+0000","last_peered":"2024-01-16T15:01:21.442573+0000","last_clean":"2024-01-16T15:01:21.442573+0000","last_became_active":"2024-01-16T15:01:19.909971+0000","last_became_peered":"2024-01-16T15:01:19.909971+0000","last_unstale":"2024-01-16T15:01:21.442573+0000","last_undegraded":"2024-01-16T15:01:21.442573+0000","last_fullsized":"2024-01-16T15:01:21.442573+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:36:47.158317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.444713+0000","last_change":"2024-01-16T15:01:18.861635+0000","last_active":"2024-01-16T15:01:21.444713+0000","last_peered":"2024-01-16T15:01:21.444713+0000","last_clean":"2024-01-16T15:01:21.444713+0000","last_became_active":"2024-01-16T15:01:18.859919+0000","last_became_peered":"2024-01-16T15:01:18.859919+0000","last_unstale":"2024-01-16T15:01:21.444713+0000","last_undegraded":"2024-01-16T15:01:21.444713+0000","last_fullsized":"2024-01-16T15:01:21.444713+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:22:02.623082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.444884+0000","last_change":"2024-01-16T15:01:19.898274+0000","last_active":"2024-01-16T15:01:21.444884+0000","last_peered":"2024-01-16T15:01:21.444884+0000","last_clean":"2024-01-16T15:01:21.444884+0000","last_became_active":"2024-01-16T15:01:19.898025+0000","last_became_peered":"2024-01-16T15:01:19.898025+0000","last_unstale":"2024-01-16T15:01:21.444884+0000","last_undegraded":"2024-01-16T15:01:21.444884+0000","last_fullsized":"2024-01-16T15:01:21.444884+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:14:12.867834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.443134+0000","last_change":"2024-01-16T15:01:18.871497+0000","last_active":"2024-01-16T15:01:21.443134+0000","last_peered":"2024-01-16T15:01:21.443134+0000","last_clean":"2024-01-16T15:01:21.443134+0000","last_became_active":"2024-01-16T15:01:18.871066+0000","last_became_peered":"2024-01-16T15:01:18.871066+0000","last_unstale":"2024-01-16T15:01:21.443134+0000","last_undegraded":"2024-01-16T15:01:21.443134+0000","last_fullsized":"2024-01-16T15:01:21.443134+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:12:28.451140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.442879+0000","last_change":"2024-01-16T15:01:19.891355+0000","last_active":"2024-01-16T15:01:21.442879+0000","last_peered":"2024-01-16T15:01:21.442879+0000","last_clean":"2024-01-16T15:01:21.442879+0000","last_became_active":"2024-01-16T15:01:19.891098+0000","last_became_peered":"2024-01-16T15:01:19.891098+0000","last_unstale":"2024-01-16T15:01:21.442879+0000","last_undegraded":"2024-01-16T15:01:21.442879+0000","last_fullsized":"2024-01-16T15:01:21.442879+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:35:31.837292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.444102+0000","last_change":"2024-01-16T15:01:18.871366+0000","last_active":"2024-01-16T15:01:21.444102+0000","last_peered":"2024-01-16T15:01:21.444102+0000","last_clean":"2024-01-16T15:01:21.444102+0000","last_became_active":"2024-01-16T15:01:18.870276+0000","last_became_peered":"2024-01-16T15:01:18.870276+0000","last_unstale":"2024-01-16T15:01:21.444102+0000","last_undegraded":"2024-01-16T15:01:21.444102+0000","last_fullsized":"2024-01-16T15:01:21.444102+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:57:03.106778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.897428+0000","last_change":"2024-01-16T15:01:19.903450+0000","last_active":"2024-01-16T15:01:20.897428+0000","last_peered":"2024-01-16T15:01:20.897428+0000","last_clean":"2024-01-16T15:01:20.897428+0000","last_became_active":"2024-01-16T15:01:19.903156+0000","last_became_peered":"2024-01-16T15:01:19.903156+0000","last_unstale":"2024-01-16T15:01:20.897428+0000","last_undegraded":"2024-01-16T15:01:20.897428+0000","last_fullsized":"2024-01-16T15:01:20.897428+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:31:08.299586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.444176+0000","last_change":"2024-01-16T15:01:18.872671+0000","last_active":"2024-01-16T15:01:21.444176+0000","last_peered":"2024-01-16T15:01:21.444176+0000","last_clean":"2024-01-16T15:01:21.444176+0000","last_became_active":"2024-01-16T15:01:18.872362+0000","last_became_peered":"2024-01-16T15:01:18.872362+0000","last_unstale":"2024-01-16T15:01:21.444176+0000","last_undegraded":"2024-01-16T15:01:21.444176+0000","last_fullsized":"2024-01-16T15:01:21.444176+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:12:28.451140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.443953+0000","last_change":"2024-01-16T15:01:19.890285+0000","last_active":"2024-01-16T15:01:21.443953+0000","last_peered":"2024-01-16T15:01:21.443953+0000","last_clean":"2024-01-16T15:01:21.443953+0000","last_became_active":"2024-01-16T15:01:19.889891+0000","last_became_peered":"2024-01-16T15:01:19.889891+0000","last_unstale":"2024-01-16T15:01:21.443953+0000","last_undegraded":"2024-01-16T15:01:21.443953+0000","last_fullsized":"2024-01-16T15:01:21.443953+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:36:47.158317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.897514+0000","last_change":"2024-01-16T15:01:18.860505+0000","last_active":"2024-01-16T15:01:20.897514+0000","last_peered":"2024-01-16T15:01:20.897514+0000","last_clean":"2024-01-16T15:01:20.897514+0000","last_became_active":"2024-01-16T15:01:18.860222+0000","last_became_peered":"2024-01-16T15:01:18.860222+0000","last_unstale":"2024-01-16T15:01:20.897514+0000","last_undegraded":"2024-01-16T15:01:20.897514+0000","last_fullsized":"2024-01-16T15:01:20.897514+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:36:46.146932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:20.903470+0000","last_change":"2024-01-16T15:01:19.896174+0000","last_active":"2024-01-16T15:01:20.903470+0000","last_peered":"2024-01-16T15:01:20.903470+0000","last_clean":"2024-01-16T15:01:20.903470+0000","last_became_active":"2024-01-16T15:01:19.894009+0000","last_became_peered":"2024-01-16T15:01:19.894009+0000","last_unstale":"2024-01-16T15:01:20.903470+0000","last_undegraded":"2024-01-16T15:01:20.903470+0000","last_fullsized":"2024-01-16T15:01:20.903470+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:18.829449+0000","last_clean_scrub_stamp":"2024-01-16T15:01:18.829449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:43:37.821973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-16T15:01:21.444478+0000","last_change":"2024-01-16T15:01:18.876295+0000","last_active":"2024-01-16T15:01:21.444478+0000","last_peered":"2024-01-16T15:01:21.444478+0000","last_clean":"2024-01-16T15:01:21.444478+0000","last_became_active":"2024-01-16T15:01:18.875885+0000","last_became_peered":"2024-01-16T15:01:18.875885+0000","last_unstale":"2024-01-16T15:01:21.444478+0000","last_undegraded":"2024-01-16T15:01:21.444478+0000","last_fullsized":"2024-01-16T15:01:21.444478+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T15:01:17.818065+0000","last_clean_scrub_stamp":"2024-01-16T15:01:17.818065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:03:48.869753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":179,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-16T15:01:19.870231+0000","last_change":"2024-01-16T14:59:53.430715+0000","last_active":"2024-01-16T15:01:19.870231+0000","last_peered":"2024-01-16T15:01:19.870231+0000","last_clean":"2024-01-16T15:01:19.870231+0000","last_became_active":"2024-01-16T14:59:53.428209+0000","last_became_peered":"2024-01-16T14:59:53.428209+0000","last_unstale":"2024-01-16T15:01:19.870231+0000","last_undegraded":"2024-01-16T15:01:19.870231+0000","last_fullsized":"2024-01-16T15:01:19.870231+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:59:52.011232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:59:52.011232+0000","last_clean_scrub_stamp":"2024-01-16T14:59:52.011232+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:57:42.594848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":983040,"data_stored":742357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 16 15:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":1.3280000000000001}]},{"osd":1,"last update":"Tue Jan 16 15:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.724,"5min":1.724,"15min":1.724},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.143,"5min":1.143,"15min":1.143},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":0.96599999999999997}]},{"osd":2,"last update":"Tue Jan 16 15:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.893,"5min":1.893,"15min":1.893},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.21,"5min":1.21,"15min":1.21},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.2110000000000001}]},{"osd":3,"last update":"Tue Jan 16 15:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"min":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.399,"5min":1.399,"15min":1.399},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.1339999999999999}]},{"osd":4,"last update":"Tue Jan 16 15:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.4889999999999999,"5min":2.4889999999999999,"15min":2.4889999999999999},"last":1.6439999999999999},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.5259999999999998,"5min":2.5259999999999998,"15min":2.5259999999999998},"last":1.409}]},{"osd":5,"last update":"Tue Jan 16 15:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.29,"15min":1.29},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.581,"5min":2.581,"15min":2.581},"last":1.716}]},{"osd":6,"last update":"Tue Jan 16 15:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1.522,"5min":1.522,"15min":1.522},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.3740000000000001,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":1.766},{"interface":"front","average":{"1min":1.415,"5min":1.415,"15min":1.415},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.0939999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.5800000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":146215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.0129999999999999}]},{"osd":1,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":1.236},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.4449999999999998,"5min":2.4449999999999998,"15min":2.4449999999999998},"last":1.4630000000000001}]},{"osd":2,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":3.0009999999999999,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":1.599},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.488,"5min":2.488,"15min":2.488},"last":1.1459999999999999}]},{"osd":3,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.373,"15min":1.373},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":4.0389999999999997,"5min":4.0389999999999997,"15min":4.0389999999999997},"last":1.514},{"interface":"front","average":{"1min":1.397,"5min":1.397,"15min":1.397},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":3.649,"5min":3.649,"15min":3.649},"last":1.427}]},{"osd":4,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.6720000000000002,"5min":2.6720000000000002,"15min":2.6720000000000002},"last":1.333},{"interface":"front","average":{"1min":1.54,"5min":1.54,"15min":1.54},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":3.7719999999999998,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":1.2849999999999999}]},{"osd":5,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":4.3840000000000003,"5min":4.3840000000000003,"15min":4.3840000000000003},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":3.3559999999999999,"5min":3.3559999999999999,"15min":3.3559999999999999},"last":1.669}]},{"osd":7,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":2.8479999999999999,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":4.2729999999999997,"5min":4.2729999999999997,"15min":4.2729999999999997},"last":1.5600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":146271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":1.026},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.80600000000000005}]},{"osd":1,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":1.6719999999999999}]},{"osd":2,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.238},{"interface":"front","average":{"1min":1.294,"5min":1.294,"15min":1.294},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.968,"5min":2.968,"15min":2.968},"last":1.508}]},{"osd":3,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":11.638,"5min":11.638,"15min":11.638},"last":1.482},{"interface":"front","average":{"1min":1.857,"5min":1.857,"15min":1.857},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":11.901999999999999,"5min":11.901999999999999,"15min":11.901999999999999},"last":1.452}]},{"osd":4,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.266,"5min":2.266,"15min":2.266},"last":1.4139999999999999}]},{"osd":6,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.7030000000000001},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.0950000000000002,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":1.5429999999999999}]},{"osd":7,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":2.3679999999999999,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":1.6020000000000001},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":2.5640000000000001,"5min":2.5640000000000001,"15min":2.5640000000000001},"last":1.639}]}]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":145703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 16 15:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.5489999999999999},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":1.0469999999999999}]},{"osd":1,"last update":"Tue Jan 16 15:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":3.0049999999999999,"5min":3.0049999999999999,"15min":3.0049999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.22,"5min":1.22,"15min":1.22},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.9470000000000001,"5min":2.9470000000000001,"15min":2.9470000000000001},"last":1.494}]},{"osd":2,"last update":"Tue Jan 16 15:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":2.137,"5min":2.137,"15min":2.137},"last":0.999},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":2.2570000000000001,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":1.373}]},{"osd":3,"last update":"Tue Jan 16 15:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":1.155},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.2250000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.28}]},{"osd":5,"last update":"Tue Jan 16 15:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.496,"15min":1.496},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":2.4020000000000001,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.454,"5min":1.454,"15min":1.454},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":1.329}]},{"osd":6,"last update":"Tue Jan 16 15:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":2.4529999999999998,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.496,"5min":2.496,"15min":2.496},"last":1.4450000000000001}]},{"osd":7,"last update":"Tue Jan 16 15:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.1960000000000002,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"min":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":2.536,"5min":2.536,"15min":2.536},"last":1.405}]}]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":153533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 16 15:00:53 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.909,"5min":1.909,"15min":1.909},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":2.863,"5min":2.863,"15min":2.863},"last":1.0029999999999999}]},{"osd":1,"last update":"Tue Jan 16 15:00:53 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.956,"5min":1.956,"15min":1.956},"last":1.51}]},{"osd":2,"last update":"Tue Jan 16 15:00:53 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.4670000000000001,"5min":2.4670000000000001,"15min":2.4670000000000001},"last":1.095},{"interface":"front","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":3.8530000000000002,"5min":3.8530000000000002,"15min":3.8530000000000002},"last":1.6850000000000001}]},{"osd":4,"last update":"Tue Jan 16 15:00:53 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":2.9950000000000001,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":1.272},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":2.6899999999999999,"5min":2.6899999999999999,"15min":2.6899999999999999},"last":1.2150000000000001}]},{"osd":5,"last update":"Tue Jan 16 15:00:53 2024","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.6509999999999998,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.8260000000000001,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":1.601}]},{"osd":6,"last update":"Tue Jan 16 15:00:53 2024","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.393,"15min":1.393},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.952,"5min":2.952,"15min":2.952},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.726,"5min":2.726,"15min":2.726},"last":1.73}]},{"osd":7,"last update":"Tue Jan 16 15:00:53 2024","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.508,"15min":1.508},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":3.9529999999999998,"5min":3.9529999999999998,"15min":3.9529999999999998},"last":2.3039999999999998},{"interface":"front","average":{"1min":1.498,"5min":1.498,"15min":1.498},"min":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"max":{"1min":2.911,"5min":2.911,"15min":2.911},"last":1.6419999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":151969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.753},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.208}]},{"osd":1,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.90200000000000002}]},{"osd":3,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":1.056},{"interface":"front","average":{"1min":1.232,"5min":1.232,"15min":1.232},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.155}]},{"osd":4,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.24,"15min":1.24},"min":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.296}]},{"osd":5,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.542,"5min":1.542,"15min":1.542},"min":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"max":{"1min":3.5030000000000001,"5min":3.5030000000000001,"15min":3.5030000000000001},"last":1.26},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"min":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":2.758,"5min":2.758,"15min":2.758},"last":1.6910000000000001}]},{"osd":6,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.476,"15min":1.476},"min":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"max":{"1min":2.1219999999999999,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.516,"5min":1.516,"15min":1.516},"min":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"max":{"1min":2.0030000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":1.4730000000000001}]},{"osd":7,"last update":"Tue Jan 16 15:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"min":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"max":{"1min":2.8620000000000001,"5min":2.8620000000000001,"15min":2.8620000000000001},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":2.2069999999999999,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":1.4319999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":147233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 16 15:00:53 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":2.5139999999999998,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":1.425},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":2.8999999999999999,"5min":2.8999999999999999,"15min":2.8999999999999999},"last":0.80000000000000004}]},{"osd":2,"last update":"Tue Jan 16 15:00:53 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.024,"5min":2.024,"15min":2.024},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":1.284}]},{"osd":3,"last update":"Tue Jan 16 15:00:53 2024","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.5019999999999998,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":5.4210000000000003,"5min":5.4210000000000003,"15min":5.4210000000000003},"last":1.202}]},{"osd":4,"last update":"Tue Jan 16 15:00:53 2024","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.177},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.4319999999999999,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":1.546}]},{"osd":5,"last update":"Tue Jan 16 15:00:53 2024","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.431,"15min":1.431},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":5.3819999999999997,"5min":5.3819999999999997,"15min":5.3819999999999997},"last":1.371},{"interface":"front","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":2.1640000000000001,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.333}]},{"osd":6,"last update":"Tue Jan 16 15:00:53 2024","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.28,"15min":1.28},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":2.2570000000000001,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":1.254}]},{"osd":7,"last update":"Tue Jan 16 15:00:53 2024","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.599},{"interface":"front","average":{"1min":1.294,"5min":1.294,"15min":1.294},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.3399999999999999,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.091}]}]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":974848,"data_stored":741248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Jan 16 15:00:55 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.5110000000000001,"5min":2.5110000000000001,"15min":2.5110000000000001},"last":1.339}]},{"osd":2,"last update":"Tue Jan 16 15:00:55 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":1.0620000000000001},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":0.61099999999999999}]},{"osd":3,"last update":"Tue Jan 16 15:00:55 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.008},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":0.96299999999999997}]},{"osd":4,"last update":"Tue Jan 16 15:00:55 2024","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.149,"15min":1.149},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.4660000000000002,"5min":2.4660000000000002,"15min":2.4660000000000002},"last":1.194},{"interface":"front","average":{"1min":1.242,"5min":1.242,"15min":1.242},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.1160000000000001,"5min":2.1160000000000001,"15min":2.1160000000000001},"last":1.4870000000000001}]},{"osd":5,"last update":"Tue Jan 16 15:00:55 2024","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.3740000000000001,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":3.23,"5min":3.23,"15min":3.23},"last":1.375}]},{"osd":6,"last update":"Tue Jan 16 15:00:55 2024","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.3580000000000001,"5min":2.3580000000000001,"15min":2.3580000000000001},"last":1.399},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":3.2890000000000001,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":1.423}]},{"osd":7,"last update":"Tue Jan 16 15:00:55 2024","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.329,"15min":1.329},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":3.3500000000000001,"5min":3.3500000000000001,"15min":3.3500000000000001},"last":1.29},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":2.621,"5min":2.621,"15min":2.621},"last":1.306}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-16T15:01:24.768 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-16T15:01:24.769 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-16T15:01:25.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.048+0000 7f12c3400640 1 Processor -- start 2024-01-16T15:01:25.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.048+0000 7f12c3400640 1 -- start start 2024-01-16T15:01:25.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.050+0000 7f12c3400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f12bc15ef00 0x7f12bc071700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:25.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.050+0000 7f12c3400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12bc0ccdf0 con 0x7f12bc141b30 2024-01-16T15:01:25.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.050+0000 7f12c3400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12bc15f470 con 0x7f12bc071c40 2024-01-16T15:01:25.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.050+0000 7f12c3400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12bc06ce80 con 0x7f12bc15ef00 2024-01-16T15:01:25.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.050+0000 7f12c1600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f12bc15ef00 0x7f12bc071700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:25.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.050+0000 7f12c2000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f12bc141b30 0x7f12bc06c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:58470/0 (socket says 172.21.15.143:58470) 2024-01-16T15:01:25.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.050+0000 7f12c2000640 1 -- 172.21.15.143:0/1927403367 learned_addr learned my addr 172.21.15.143:0/1927403367 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:01:25.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.050+0000 7f12c1600640 1 -- 172.21.15.143:0/1927403367 >> v1:172.21.15.204:6789/0 conn(0x7f12bc071c40 legacy=0x7f12bc0cede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:25.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.050+0000 7f12c1600640 1 -- 172.21.15.143:0/1927403367 >> v1:172.21.15.143:6789/0 conn(0x7f12bc141b30 legacy=0x7f12bc06c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:25.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.050+0000 7f12c1600640 1 -- 172.21.15.143:0/1927403367 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12bc15f750 con 0x7f12bc15ef00 2024-01-16T15:01:25.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.052+0000 7f12c1600640 1 --2- 172.21.15.143:0/1927403367 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12bc15ef00 0x7f12bc071700 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f12a40097a0 tx=0x7f12a402b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f454d1040787c78 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:25.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.052+0000 7f12c0c00640 1 -- 172.21.15.143:0/1927403367 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a402f050 con 0x7f12bc15ef00 2024-01-16T15:01:25.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.052+0000 7f12c0c00640 1 -- 172.21.15.143:0/1927403367 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a4031700 con 0x7f12bc15ef00 2024-01-16T15:01:25.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.052+0000 7f12c0c00640 1 -- 172.21.15.143:0/1927403367 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a4031a90 con 0x7f12bc15ef00 2024-01-16T15:01:25.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.053+0000 7f12c3400640 1 -- 172.21.15.143:0/1927403367 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12bc15ef00 msgr2=0x7f12bc071700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:25.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.053+0000 7f12c3400640 1 --2- 172.21.15.143:0/1927403367 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12bc15ef00 0x7f12bc071700 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f12a40097a0 tx=0x7f12a402b7a0 comp rx=0 tx=0).stop 2024-01-16T15:01:25.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.053+0000 7f12c3400640 1 -- 172.21.15.143:0/1927403367 shutdown_connections 2024-01-16T15:01:25.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.053+0000 7f12c3400640 1 --2- 172.21.15.143:0/1927403367 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12bc15ef00 0x7f12bc071700 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:25.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.053+0000 7f12c3400640 1 -- 172.21.15.143:0/1927403367 >> 172.21.15.143:0/1927403367 conn(0x7f12bc06a500 msgr2=0x7f12bc06a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:25.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.053+0000 7f12c3400640 1 -- 172.21.15.143:0/1927403367 shutdown_connections 2024-01-16T15:01:25.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.053+0000 7f12c3400640 1 -- 172.21.15.143:0/1927403367 wait complete. 2024-01-16T15:01:25.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.055+0000 7f12c3400640 1 Processor -- start 2024-01-16T15:01:25.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.055+0000 7f12c3400640 1 -- start start 2024-01-16T15:01:25.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.057+0000 7f12c3400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f12bc071c40 0x7f12bc0b3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:25.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.057+0000 7f12c3400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12bc141b30 0x7f12bc140450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:25.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.057+0000 7f12c3400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12bc15ef00 0x7f12bc0b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:25.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.057+0000 7f12c3400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12bc0b4020 con 0x7f12bc141b30 2024-01-16T15:01:25.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.057+0000 7f12c3400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12bc0b3650 con 0x7f12bc15ef00 2024-01-16T15:01:25.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.057+0000 7f12c3400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12bc0b2a40 con 0x7f12bc071c40 2024-01-16T15:01:25.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.057+0000 7f12c1600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12bc141b30 0x7f12bc140450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:25.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.057+0000 7f12c1600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12bc141b30 0x7f12bc140450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40904/0 (socket says 172.21.15.143:40904) 2024-01-16T15:01:25.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.057+0000 7f12c1600640 1 -- 172.21.15.143:0/2837639747 learned_addr learned my addr 172.21.15.143:0/2837639747 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:25.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.057+0000 7f12c2000640 1 --2- 172.21.15.143:0/2837639747 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f12bc071c40 0x7f12bc0b3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:25.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.057+0000 7f12c2a00640 1 --2- 172.21.15.143:0/2837639747 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12bc15ef00 0x7f12bc0b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:25.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.057+0000 7f12c1600640 1 -- 172.21.15.143:0/2837639747 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f12bc071c40 msgr2=0x7f12bc0b3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:25.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.057+0000 7f12c1600640 1 --2- 172.21.15.143:0/2837639747 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f12bc071c40 0x7f12bc0b3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:25.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.057+0000 7f12c1600640 1 -- 172.21.15.143:0/2837639747 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12bc15ef00 msgr2=0x7f12bc0b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:25.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.057+0000 7f12c1600640 1 --2- 172.21.15.143:0/2837639747 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12bc15ef00 0x7f12bc0b1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:25.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.057+0000 7f12c1600640 1 -- 172.21.15.143:0/2837639747 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a4009400 con 0x7f12bc141b30 2024-01-16T15:01:25.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.057+0000 7f12c2a00640 1 --2- 172.21.15.143:0/2837639747 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12bc15ef00 0x7f12bc0b1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:01:25.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.058+0000 7f12c1600640 1 --2- 172.21.15.143:0/2837639747 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12bc141b30 0x7f12bc140450 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f12a40098c0 tx=0x7f12a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:25.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.058+0000 7f12ba000640 1 -- 172.21.15.143:0/2837639747 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a4033c30 con 0x7f12bc141b30 2024-01-16T15:01:25.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.058+0000 7f12ba000640 1 -- 172.21.15.143:0/2837639747 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a4033d90 con 0x7f12bc141b30 2024-01-16T15:01:25.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.058+0000 7f12c3400640 1 -- 172.21.15.143:0/2837639747 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12bc0b2d20 con 0x7f12bc141b30 2024-01-16T15:01:25.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.058+0000 7f12ba000640 1 -- 172.21.15.143:0/2837639747 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a403c640 con 0x7f12bc141b30 2024-01-16T15:01:25.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.058+0000 7f12c3400640 1 -- 172.21.15.143:0/2837639747 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12bc0b32e0 con 0x7f12bc141b30 2024-01-16T15:01:25.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.060+0000 7f12c3400640 1 -- 172.21.15.143:0/2837639747 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12bc0b37c0 con 0x7f12bc141b30 2024-01-16T15:01:25.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.060+0000 7f12ba000640 1 -- 172.21.15.143:0/2837639747 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f12a403a040 con 0x7f12bc141b30 2024-01-16T15:01:25.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.061+0000 7f12ba000640 1 --2- 172.21.15.143:0/2837639747 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f129406b2f0 0x7f129406d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:25.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.061+0000 7f12ba000640 1 -- 172.21.15.143:0/2837639747 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f12a40ab7b0 con 0x7f12bc141b30 2024-01-16T15:01:25.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.062+0000 7f12c2000640 1 --2- 172.21.15.143:0/2837639747 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f129406b2f0 0x7f129406d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:25.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.062+0000 7f12c2000640 1 --2- 172.21.15.143:0/2837639747 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f129406b2f0 0x7f129406d7b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f12ac005880 tx=0x7f12ac009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:25.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.066+0000 7f12ba000640 1 -- 172.21.15.143:0/2837639747 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f12a40764f0 con 0x7f12bc141b30 2024-01-16T15:01:25.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.325+0000 7f12c3400640 1 -- 172.21.15.143:0/2837639747 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f12bc04e840 con 0x7f12bc141b30 2024-01-16T15:01:25.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.330+0000 7f12ba000640 1 -- 172.21.15.143:0/2837639747 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) v1 ==== 190+0+147 (secure 0 0 0) 0x7f12a40430e0 con 0x7f12bc141b30 2024-01-16T15:01:25.310 INFO:teuthology.orchestra.run.smithi143.stdout:[client.0] 2024-01-16T15:01:25.310 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQCxmaZlyhubNxAANh0FPLUzeCMMZcrjC6cocw== 2024-01-16T15:01:25.310 INFO:teuthology.orchestra.run.smithi143.stdout: caps mds = "allow" 2024-01-16T15:01:25.310 INFO:teuthology.orchestra.run.smithi143.stdout: caps mon = "allow r" 2024-01-16T15:01:25.310 INFO:teuthology.orchestra.run.smithi143.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-01-16T15:01:25.310 INFO:teuthology.orchestra.run.smithi143.stderr:updated caps for client.0 2024-01-16T15:01:25.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.333+0000 7f12c3400640 1 -- 172.21.15.143:0/2837639747 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f129406b2f0 msgr2=0x7f129406d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:25.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.333+0000 7f12c3400640 1 --2- 172.21.15.143:0/2837639747 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f129406b2f0 0x7f129406d7b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f12ac005880 tx=0x7f12ac009040 comp rx=0 tx=0).stop 2024-01-16T15:01:25.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.333+0000 7f12c3400640 1 -- 172.21.15.143:0/2837639747 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12bc141b30 msgr2=0x7f12bc140450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:25.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.333+0000 7f12c3400640 1 --2- 172.21.15.143:0/2837639747 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12bc141b30 0x7f12bc140450 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f12a40098c0 tx=0x7f12a40046c0 comp rx=0 tx=0).stop 2024-01-16T15:01:25.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.333+0000 7f12c3400640 1 -- 172.21.15.143:0/2837639747 shutdown_connections 2024-01-16T15:01:25.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.333+0000 7f12c3400640 1 --2- 172.21.15.143:0/2837639747 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f129406b2f0 0x7f129406d7b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:25.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.333+0000 7f12c3400640 1 --2- 172.21.15.143:0/2837639747 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12bc15ef00 0x7f12bc0b1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:25.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.333+0000 7f12c3400640 1 --2- 172.21.15.143:0/2837639747 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12bc141b30 0x7f12bc140450 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:25.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.333+0000 7f12c3400640 1 --2- 172.21.15.143:0/2837639747 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f12bc071c40 0x7f12bc0b3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:25.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.333+0000 7f12c3400640 1 -- 172.21.15.143:0/2837639747 >> 172.21.15.143:0/2837639747 conn(0x7f12bc06a500 msgr2=0x7f12bc15c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:25.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.333+0000 7f12c3400640 1 -- 172.21.15.143:0/2837639747 shutdown_connections 2024-01-16T15:01:25.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.333+0000 7f12c3400640 1 -- 172.21.15.143:0/2837639747 wait complete. 2024-01-16T15:01:25.329 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-16T15:01:25.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.559+0000 7f3758600640 1 Processor -- start 2024-01-16T15:01:25.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.559+0000 7f3758600640 1 -- start start 2024-01-16T15:01:25.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.560+0000 7f3758600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f37500b2250 0x7f375015df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:25.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.560+0000 7f3758600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37500b2630 0x7f37500ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:25.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.560+0000 7f3758600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f37500cd470 0x7f375015b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:25.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.560+0000 7f3758600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f375015a990 con 0x7f37500cd470 2024-01-16T15:01:25.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.560+0000 7f3758600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f375015ab80 con 0x7f37500b2250 2024-01-16T15:01:25.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.560+0000 7f3758600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f375015e680 con 0x7f37500b2630 2024-01-16T15:01:25.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.561+0000 7f3756600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f37500cd470 0x7f375015b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:25.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.561+0000 7f3756600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f37500cd470 0x7f375015b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40916/0 (socket says 172.21.15.143:40916) 2024-01-16T15:01:25.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.561+0000 7f3756600640 1 -- 172.21.15.143:0/4278310542 learned_addr learned my addr 172.21.15.143:0/4278310542 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:25.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.561+0000 7f3755c00640 1 --2- 172.21.15.143:0/4278310542 >> v2:172.21.15.204:3300/0 conn(0x7f37500b2250 0x7f375015df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:25.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.561+0000 7f3755200640 1 --2- 172.21.15.143:0/4278310542 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37500b2630 0x7f37500ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:25.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.561+0000 7f3756600640 1 -- 172.21.15.143:0/4278310542 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37500b2630 msgr2=0x7f37500ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:25.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.561+0000 7f3756600640 1 --2- 172.21.15.143:0/4278310542 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37500b2630 0x7f37500ccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:25.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.561+0000 7f3756600640 1 -- 172.21.15.143:0/4278310542 >> v2:172.21.15.204:3300/0 conn(0x7f37500b2250 msgr2=0x7f375015df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:25.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.561+0000 7f3756600640 1 --2- 172.21.15.143:0/4278310542 >> v2:172.21.15.204:3300/0 conn(0x7f37500b2250 0x7f375015df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:25.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.561+0000 7f3756600640 1 -- 172.21.15.143:0/4278310542 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375015e960 con 0x7f37500cd470 2024-01-16T15:01:25.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.561+0000 7f3755c00640 1 --2- 172.21.15.143:0/4278310542 >> v2:172.21.15.204:3300/0 conn(0x7f37500b2250 0x7f375015df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:01:25.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.562+0000 7f3756600640 1 --2- 172.21.15.143:0/4278310542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37500cd470 0x7f375015b080 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3750072a70 tx=0x7f374802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c74d54d12ec4d456 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:25.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.562+0000 7f374fe00640 1 -- 172.21.15.143:0/4278310542 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374802ca70 con 0x7f37500cd470 2024-01-16T15:01:25.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.563+0000 7f374fe00640 1 -- 172.21.15.143:0/4278310542 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f374802cbd0 con 0x7f37500cd470 2024-01-16T15:01:25.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.563+0000 7f374fe00640 1 -- 172.21.15.143:0/4278310542 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37480343f0 con 0x7f37500cd470 2024-01-16T15:01:25.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.563+0000 7f3758600640 1 -- 172.21.15.143:0/4278310542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37500cd470 msgr2=0x7f375015b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:25.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.563+0000 7f3758600640 1 --2- 172.21.15.143:0/4278310542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37500cd470 0x7f375015b080 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3750072a70 tx=0x7f374802bed0 comp rx=0 tx=0).stop 2024-01-16T15:01:25.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.563+0000 7f3758600640 1 -- 172.21.15.143:0/4278310542 shutdown_connections 2024-01-16T15:01:25.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.563+0000 7f3758600640 1 --2- 172.21.15.143:0/4278310542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37500cd470 0x7f375015b080 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:25.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.563+0000 7f3758600640 1 --2- 172.21.15.143:0/4278310542 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37500b2630 0x7f37500ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:25.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.563+0000 7f3758600640 1 --2- 172.21.15.143:0/4278310542 >> v2:172.21.15.204:3300/0 conn(0x7f37500b2250 0x7f375015df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:25.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.563+0000 7f3758600640 1 -- 172.21.15.143:0/4278310542 >> 172.21.15.143:0/4278310542 conn(0x7f375006a920 msgr2=0x7f375006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:25.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.563+0000 7f3758600640 1 -- 172.21.15.143:0/4278310542 shutdown_connections 2024-01-16T15:01:25.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.563+0000 7f3758600640 1 -- 172.21.15.143:0/4278310542 wait complete. 2024-01-16T15:01:25.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.565+0000 7f3758600640 1 Processor -- start 2024-01-16T15:01:25.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.565+0000 7f3758600640 1 -- start start 2024-01-16T15:01:25.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.566+0000 7f3758600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37500b2250 0x7f37501574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:25.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.566+0000 7f3758600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37500b2630 0x7f37500b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:25.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.566+0000 7f3758600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37500cd470 0x7f37501f2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:25.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.566+0000 7f3758600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3750158040 con 0x7f37500b2630 2024-01-16T15:01:25.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.566+0000 7f3758600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37501efd80 con 0x7f37500b2250 2024-01-16T15:01:25.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.566+0000 7f3758600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37500b1a30 con 0x7f37500cd470 2024-01-16T15:01:25.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.566+0000 7f3755200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37500b2630 0x7f37500b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:25.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.567+0000 7f3755200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37500b2630 0x7f37500b14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40922/0 (socket says 172.21.15.143:40922) 2024-01-16T15:01:25.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.567+0000 7f3755200640 1 -- 172.21.15.143:0/2586080134 learned_addr learned my addr 172.21.15.143:0/2586080134 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:25.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.567+0000 7f3756600640 1 --2- 172.21.15.143:0/2586080134 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37500cd470 0x7f37501f2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:25.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.567+0000 7f3755c00640 1 --2- 172.21.15.143:0/2586080134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37500b2250 0x7f37501574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:25.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.567+0000 7f3755200640 1 -- 172.21.15.143:0/2586080134 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37500cd470 msgr2=0x7f37501f2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:25.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.567+0000 7f3755200640 1 --2- 172.21.15.143:0/2586080134 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37500cd470 0x7f37501f2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:25.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.567+0000 7f3755200640 1 -- 172.21.15.143:0/2586080134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37500b2250 msgr2=0x7f37501574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:25.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.567+0000 7f3755200640 1 --2- 172.21.15.143:0/2586080134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37500b2250 0x7f37501574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:25.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.567+0000 7f3755200640 1 -- 172.21.15.143:0/2586080134 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3748009b30 con 0x7f37500b2630 2024-01-16T15:01:25.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.567+0000 7f3755200640 1 --2- 172.21.15.143:0/2586080134 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37500b2630 0x7f37500b14f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f373800c7f0 tx=0x7f373800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:25.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.568+0000 7f374d600640 1 -- 172.21.15.143:0/2586080134 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3738004220 con 0x7f37500b2630 2024-01-16T15:01:25.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.568+0000 7f374d600640 1 -- 172.21.15.143:0/2586080134 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37380043a0 con 0x7f37500b2630 2024-01-16T15:01:25.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.568+0000 7f374d600640 1 -- 172.21.15.143:0/2586080134 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3738007860 con 0x7f37500b2630 2024-01-16T15:01:25.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.568+0000 7f3758600640 1 -- 172.21.15.143:0/2586080134 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3750157a80 con 0x7f37500b2630 2024-01-16T15:01:25.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.568+0000 7f3758600640 1 -- 172.21.15.143:0/2586080134 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3750070760 con 0x7f37500b2630 2024-01-16T15:01:25.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.569+0000 7f3758600640 1 -- 172.21.15.143:0/2586080134 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37500705c0 con 0x7f37500b2630 2024-01-16T15:01:25.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.570+0000 7f374d600640 1 -- 172.21.15.143:0/2586080134 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f3738007bd0 con 0x7f37500b2630 2024-01-16T15:01:25.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.571+0000 7f374d600640 1 --2- 172.21.15.143:0/2586080134 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f372806b2c0 0x7f372806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:25.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.571+0000 7f374d600640 1 -- 172.21.15.143:0/2586080134 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3738089ca0 con 0x7f37500b2630 2024-01-16T15:01:25.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.571+0000 7f3755c00640 1 --2- 172.21.15.143:0/2586080134 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f372806b2c0 0x7f372806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:25.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.572+0000 7f3755c00640 1 --2- 172.21.15.143:0/2586080134 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f372806b2c0 0x7f372806d780 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3740006b10 tx=0x7f3740008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:25.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.575+0000 7f374d600640 1 -- 172.21.15.143:0/2586080134 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f37500705c0 con 0x7f37500b2630 2024-01-16T15:01:25.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.805+0000 7f3758600640 1 -- 172.21.15.143:0/2586080134 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37500705c0 con 0x7f37500b2630 2024-01-16T15:01:25.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.807+0000 7f374d600640 1 -- 172.21.15.143:0/2586080134 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f37500705c0 con 0x7f37500b2630 2024-01-16T15:01:25.787 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:01:25.788 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4732,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1793346439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1793346439},{"type":"v1","addr":"172.21.15.204:6835","nonce":1793346439}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5082,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/524822971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":524822971},{"type":"v1","addr":"172.21.15.204:6833","nonce":524822971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5085,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/831847190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":831847190},{"type":"v1","addr":"172.21.15.143:6835","nonce":831847190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-16T15:01:19.290571+0000","modified":"2024-01-16T15:01:22.368828+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5094},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5094":{"gid":5094,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.143:6837/4104468169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4104468169},{"type":"v1","addr":"172.21.15.143:6837","nonce":4104468169}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-01-16T15:01:25.788 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 17 2024-01-16T15:01:25.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.809+0000 7f3758600640 1 -- 172.21.15.143:0/2586080134 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f372806b2c0 msgr2=0x7f372806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:25.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.809+0000 7f3758600640 1 --2- 172.21.15.143:0/2586080134 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f372806b2c0 0x7f372806d780 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3740006b10 tx=0x7f3740008040 comp rx=0 tx=0).stop 2024-01-16T15:01:25.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.809+0000 7f3758600640 1 -- 172.21.15.143:0/2586080134 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37500b2630 msgr2=0x7f37500b14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:25.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.809+0000 7f3758600640 1 --2- 172.21.15.143:0/2586080134 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37500b2630 0x7f37500b14f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f373800c7f0 tx=0x7f373800ccb0 comp rx=0 tx=0).stop 2024-01-16T15:01:25.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.810+0000 7f3758600640 1 -- 172.21.15.143:0/2586080134 shutdown_connections 2024-01-16T15:01:25.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.810+0000 7f3758600640 1 --2- 172.21.15.143:0/2586080134 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f372806b2c0 0x7f372806d780 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:25.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.810+0000 7f3758600640 1 --2- 172.21.15.143:0/2586080134 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37500cd470 0x7f37501f2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:25.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.810+0000 7f3758600640 1 --2- 172.21.15.143:0/2586080134 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37500b2630 0x7f37500b14f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:25.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.810+0000 7f3758600640 1 --2- 172.21.15.143:0/2586080134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37500b2250 0x7f37501574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:25.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.810+0000 7f3758600640 1 -- 172.21.15.143:0/2586080134 >> 172.21.15.143:0/2586080134 conn(0x7f375006a920 msgr2=0x7f37500cf420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:25.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.810+0000 7f3758600640 1 -- 172.21.15.143:0/2586080134 shutdown_connections 2024-01-16T15:01:25.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:25.810+0000 7f3758600640 1 -- 172.21.15.143:0/2586080134 wait complete. 2024-01-16T15:01:25.793 INFO:tasks.ceph.mds.a.smithi143.stderr:2024-01-16T15:01:25.813+0000 7fcc8c400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-16T15:01:26.212 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-16T15:01:19.290571+0000', 'modified': '2024-01-16T15:01:22.368828+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5094}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5094': {'gid': 5094, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.143:6837/4104468169', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6836', 'nonce': 4104468169}, {'type': 'v1', 'addr': '172.21.15.143:6837', 'nonce': 4104468169}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-01-16T15:01:26.212 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-16T15:01:26.212 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-01-16T15:01:26.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.452+0000 7f4f51c00640 1 Processor -- start 2024-01-16T15:01:26.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.453+0000 7f4f51c00640 1 -- start start 2024-01-16T15:01:26.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.454+0000 7f4f51c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4f4c0c59b0 0x7f4f4c166230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:26.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.454+0000 7f4f51c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f4c0c5dc0 con 0x7f4f4c0cc0f0 2024-01-16T15:01:26.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.454+0000 7f4f51c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f4c15eea0 con 0x7f4f4c15b440 2024-01-16T15:01:26.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.454+0000 7f4f51c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f4c15b820 con 0x7f4f4c0c59b0 2024-01-16T15:01:26.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.454+0000 7f4f4a000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f4f4c0cc0f0 0x7f4f4c0c52a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:58486/0 (socket says 172.21.15.143:58486) 2024-01-16T15:01:26.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.454+0000 7f4f4a000640 1 -- 172.21.15.143:0/4002747381 learned_addr learned my addr 172.21.15.143:0/4002747381 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:01:26.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.455+0000 7f4f4b400640 1 --2- 172.21.15.143:0/4002747381 >> v2:172.21.15.204:3300/0 conn(0x7f4f4c0c59b0 0x7f4f4c166230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:26.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.455+0000 7f4f49600640 1 -- 172.21.15.143:0/4002747381 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1541801448 0 0) 0x7f4f4c0c5dc0 con 0x7f4f4c0cc0f0 2024-01-16T15:01:26.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.455+0000 7f4f49600640 1 -- 172.21.15.143:0/4002747381 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f2c003520 con 0x7f4f4c0cc0f0 2024-01-16T15:01:26.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.455+0000 7f4f49600640 1 -- 172.21.15.143:0/4002747381 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2828658385 0 0) 0x7f4f2c003520 con 0x7f4f4c0cc0f0 2024-01-16T15:01:26.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.455+0000 7f4f49600640 1 -- 172.21.15.143:0/4002747381 >> v2:172.21.15.204:3300/0 conn(0x7f4f4c0c59b0 msgr2=0x7f4f4c166230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:26.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.455+0000 7f4f49600640 1 --2- 172.21.15.143:0/4002747381 >> v2:172.21.15.204:3300/0 conn(0x7f4f4c0c59b0 0x7f4f4c166230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.455+0000 7f4f49600640 1 -- 172.21.15.143:0/4002747381 >> v1:172.21.15.204:6789/0 conn(0x7f4f4c15b440 legacy=0x7f4f4c0cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:26.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.455+0000 7f4f49600640 1 -- 172.21.15.143:0/4002747381 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f4c06d600 con 0x7f4f4c0cc0f0 2024-01-16T15:01:26.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.455+0000 7f4f49600640 1 -- 172.21.15.143:0/4002747381 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7f4f34001670 con 0x7f4f4c0cc0f0 2024-01-16T15:01:26.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.455+0000 7f4f49600640 1 -- 172.21.15.143:0/4002747381 >> v1:172.21.15.143:6789/0 conn(0x7f4f4c0cc0f0 legacy=0x7f4f4c0c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:26.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.455+0000 7f4f49600640 1 --2- 172.21.15.143:0/4002747381 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f2c003970 0x7f4f2c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:26.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.456+0000 7f4f49600640 1 --2- 172.21.15.143:0/4002747381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f2c0047c0 0x7f4f2c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:26.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.456+0000 7f4f49600640 1 --2- 172.21.15.143:0/4002747381 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f2c005180 0x7f4f2c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:26.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.456+0000 7f4f49600640 1 -- 172.21.15.143:0/4002747381 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4c15b820 con 0x7f4f2c003970 2024-01-16T15:01:26.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.456+0000 7f4f49600640 1 -- 172.21.15.143:0/4002747381 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2c00c120 con 0x7f4f2c0047c0 2024-01-16T15:01:26.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.456+0000 7f4f49600640 1 -- 172.21.15.143:0/4002747381 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2c00c2e0 con 0x7f4f2c005180 2024-01-16T15:01:26.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.456+0000 7f4f4aa00640 1 --2- 172.21.15.143:0/4002747381 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f2c003970 0x7f4f2c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:26.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.456+0000 7f4f4a000640 1 --2- 172.21.15.143:0/4002747381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f2c0047c0 0x7f4f2c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:26.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.456+0000 7f4f4aa00640 1 -- 172.21.15.143:0/4002747381 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f2c005180 msgr2=0x7f4f2c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:26.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.456+0000 7f4f4aa00640 1 --2- 172.21.15.143:0/4002747381 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f2c005180 0x7f4f2c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.456+0000 7f4f4aa00640 1 -- 172.21.15.143:0/4002747381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f2c0047c0 msgr2=0x7f4f2c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:26.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.456+0000 7f4f4aa00640 1 --2- 172.21.15.143:0/4002747381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f2c0047c0 0x7f4f2c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.456+0000 7f4f4aa00640 1 -- 172.21.15.143:0/4002747381 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f2c003520 con 0x7f4f2c003970 2024-01-16T15:01:26.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.456+0000 7f4f4b400640 1 -- 172.21.15.143:0/4002747381 reap_dead start 2024-01-16T15:01:26.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.457+0000 7f4f4aa00640 1 --2- 172.21.15.143:0/4002747381 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f2c003970 0x7f4f2c003d90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c001750 tx=0x7f4f2c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5366b044a7e35d72 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:26.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.458+0000 7f4f49600640 1 -- 172.21.15.143:0/4002747381 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c0052d0 con 0x7f4f2c003970 2024-01-16T15:01:26.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.458+0000 7f4f49600640 1 -- 172.21.15.143:0/4002747381 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f3c02e050 con 0x7f4f2c003970 2024-01-16T15:01:26.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.458+0000 7f4f49600640 1 -- 172.21.15.143:0/4002747381 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c003830 con 0x7f4f2c003970 2024-01-16T15:01:26.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.458+0000 7f4f51c00640 1 -- 172.21.15.143:0/4002747381 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f2c003970 msgr2=0x7f4f2c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:26.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.458+0000 7f4f51c00640 1 --2- 172.21.15.143:0/4002747381 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f2c003970 0x7f4f2c003d90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c001750 tx=0x7f4f2c0057f0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.458+0000 7f4f51c00640 1 -- 172.21.15.143:0/4002747381 shutdown_connections 2024-01-16T15:01:26.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.458+0000 7f4f51c00640 1 --2- 172.21.15.143:0/4002747381 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f2c003970 0x7f4f2c003d90 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.458+0000 7f4f51c00640 1 -- 172.21.15.143:0/4002747381 >> 172.21.15.143:0/4002747381 conn(0x7f4f4c0b0c50 msgr2=0x7f4f4c0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:26.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.458+0000 7f4f51c00640 1 -- 172.21.15.143:0/4002747381 shutdown_connections 2024-01-16T15:01:26.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.458+0000 7f4f51c00640 1 -- 172.21.15.143:0/4002747381 wait complete. 2024-01-16T15:01:26.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.460+0000 7f4f51c00640 1 Processor -- start 2024-01-16T15:01:26.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.460+0000 7f4f51c00640 1 -- start start 2024-01-16T15:01:26.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.461+0000 7f4f51c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f2c003970 0x7f4f4c1ee4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:26.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.461+0000 7f4f51c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f4c146510 0x7f4f4c146930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:26.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.461+0000 7f4f51c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f4c146e70 0x7f4f4c147670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:26.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.461+0000 7f4f51c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4c146080 con 0x7f4f2c003970 2024-01-16T15:01:26.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.461+0000 7f4f51c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4c145320 con 0x7f4f4c146e70 2024-01-16T15:01:26.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.461+0000 7f4f51c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4c1eea10 con 0x7f4f4c146510 2024-01-16T15:01:26.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.461+0000 7f4f4aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f2c003970 0x7f4f4c1ee4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:26.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.461+0000 7f4f4aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f2c003970 0x7f4f4c1ee4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40932/0 (socket says 172.21.15.143:40932) 2024-01-16T15:01:26.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.461+0000 7f4f4aa00640 1 -- 172.21.15.143:0/3142647184 learned_addr learned my addr 172.21.15.143:0/3142647184 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:26.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.462+0000 7f4f4a000640 1 --2- 172.21.15.143:0/3142647184 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f4c146510 0x7f4f4c146930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:26.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.462+0000 7f4f4b400640 1 --2- 172.21.15.143:0/3142647184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f4c146e70 0x7f4f4c147670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:26.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.462+0000 7f4f4aa00640 1 -- 172.21.15.143:0/3142647184 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f4c146510 msgr2=0x7f4f4c146930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:26.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.462+0000 7f4f4aa00640 1 --2- 172.21.15.143:0/3142647184 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f4c146510 0x7f4f4c146930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.462+0000 7f4f4aa00640 1 -- 172.21.15.143:0/3142647184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f4c146e70 msgr2=0x7f4f4c147670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:26.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.462+0000 7f4f4aa00640 1 --2- 172.21.15.143:0/3142647184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f4c146e70 0x7f4f4c147670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.462+0000 7f4f4aa00640 1 -- 172.21.15.143:0/3142647184 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f4c15e840 con 0x7f4f2c003970 2024-01-16T15:01:26.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.462+0000 7f4f4aa00640 1 --2- 172.21.15.143:0/3142647184 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f2c003970 0x7f4f4c1ee4d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c02bc30 tx=0x7f4f3c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:26.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.462+0000 7f4f42a00640 1 -- 172.21.15.143:0/3142647184 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c0076c0 con 0x7f4f2c003970 2024-01-16T15:01:26.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.462+0000 7f4f42a00640 1 -- 172.21.15.143:0/3142647184 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f3c007a30 con 0x7f4f2c003970 2024-01-16T15:01:26.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.463+0000 7f4f42a00640 1 -- 172.21.15.143:0/3142647184 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c006be0 con 0x7f4f2c003970 2024-01-16T15:01:26.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.463+0000 7f4f51c00640 1 -- 172.21.15.143:0/3142647184 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f4c09c180 con 0x7f4f2c003970 2024-01-16T15:01:26.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.463+0000 7f4f51c00640 1 -- 172.21.15.143:0/3142647184 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f4c09c740 con 0x7f4f2c003970 2024-01-16T15:01:26.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.465+0000 7f4f42a00640 1 -- 172.21.15.143:0/3142647184 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f4f3c006d40 con 0x7f4f2c003970 2024-01-16T15:01:26.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.466+0000 7f4f42a00640 1 --2- 172.21.15.143:0/3142647184 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f4f2406b2f0 0x7f4f2406d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:26.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.466+0000 7f4f42a00640 1 -- 172.21.15.143:0/3142647184 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4f3c0ab7f0 con 0x7f4f2c003970 2024-01-16T15:01:26.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.466+0000 7f4f4a000640 1 --2- 172.21.15.143:0/3142647184 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f4f2406b2f0 0x7f4f2406d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:26.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.467+0000 7f4f549fd740 1 Processor -- start 2024-01-16T15:01:26.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.467+0000 7f4f4a000640 1 --2- 172.21.15.143:0/3142647184 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f4f2406b2f0 0x7f4f2406d7b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4f340018e0 tx=0x7f4f34009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:26.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.467+0000 7f4f549fd740 1 -- start start 2024-01-16T15:01:26.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.468+0000 7f4f549fd740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f09a842ea0 0x55f09a843280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:26.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.468+0000 7f4f549fd740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55f09a83dba0 0x55f09a8420a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:26.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.468+0000 7f4f549fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a8425e0 0x55f09a886560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:26.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.468+0000 7f4f549fd740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f09a7a1880 con 0x55f09a842ea0 2024-01-16T15:01:26.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.468+0000 7f4f549fd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f09a7b0ae0 con 0x55f09a8425e0 2024-01-16T15:01:26.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.468+0000 7f4f549fd740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f09a7ae1d0 con 0x55f09a83dba0 2024-01-16T15:01:26.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.468+0000 7f4f4aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f09a842ea0 0x55f09a843280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:26.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.468+0000 7f4f4aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f09a842ea0 0x55f09a843280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40948/0 (socket says 172.21.15.143:40948) 2024-01-16T15:01:26.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.468+0000 7f4f4aa00640 1 -- 172.21.15.143:0/3704751321 learned_addr learned my addr 172.21.15.143:0/3704751321 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:26.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.468+0000 7f4f4aa00640 1 -- 172.21.15.143:0/3704751321 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55f09a83dba0 msgr2=0x55f09a8420a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:26.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.468+0000 7f4f4aa00640 1 --2- 172.21.15.143:0/3704751321 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55f09a83dba0 0x55f09a8420a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.468+0000 7f4f4aa00640 1 -- 172.21.15.143:0/3704751321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a8425e0 msgr2=0x55f09a886560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:26.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.468+0000 7f4f4aa00640 1 --2- 172.21.15.143:0/3704751321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a8425e0 0x55f09a886560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.468+0000 7f4f4aa00640 1 -- 172.21.15.143:0/3704751321 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f09a771860 con 0x55f09a842ea0 2024-01-16T15:01:26.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.469+0000 7f4f4aa00640 1 --2- 172.21.15.143:0/3704751321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f09a842ea0 0x55f09a843280 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c07a180 tx=0x7f4f3c031f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d0061a3f44d275 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:26.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.469+0000 7f4f3b400640 1 -- 172.21.15.143:0/3704751321 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c07e160 con 0x55f09a842ea0 2024-01-16T15:01:26.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.469+0000 7f4f3b400640 1 -- 172.21.15.143:0/3704751321 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f3c07e340 con 0x55f09a842ea0 2024-01-16T15:01:26.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.469+0000 7f4f3b400640 1 -- 172.21.15.143:0/3704751321 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c082580 con 0x55f09a842ea0 2024-01-16T15:01:26.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.469+0000 7f4f549fd740 1 -- 172.21.15.143:0/3704751321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f09a842ea0 msgr2=0x55f09a843280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:26.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.469+0000 7f4f549fd740 1 --2- 172.21.15.143:0/3704751321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f09a842ea0 0x55f09a843280 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c07a180 tx=0x7f4f3c031f40 comp rx=0 tx=0).stop 2024-01-16T15:01:26.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.469+0000 7f4f549fd740 1 -- 172.21.15.143:0/3704751321 shutdown_connections 2024-01-16T15:01:26.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.469+0000 7f4f549fd740 1 --2- 172.21.15.143:0/3704751321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a8425e0 0x55f09a886560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.469+0000 7f4f549fd740 1 --2- 172.21.15.143:0/3704751321 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55f09a83dba0 0x55f09a8420a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.469+0000 7f4f549fd740 1 --2- 172.21.15.143:0/3704751321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f09a842ea0 0x55f09a843280 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.469+0000 7f4f549fd740 1 -- 172.21.15.143:0/3704751321 >> 172.21.15.143:0/3704751321 conn(0x55f09a804c70 msgr2=0x55f09a888990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:26.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.470+0000 7f4f549fd740 1 -- 172.21.15.143:0/3704751321 shutdown_connections 2024-01-16T15:01:26.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.470+0000 7f4f549fd740 1 -- 172.21.15.143:0/3704751321 wait complete. 2024-01-16T15:01:26.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.470+0000 7f4f549fd740 1 -- start start 2024-01-16T15:01:26.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.471+0000 7f4f549fd740 1 Processor -- start 2024-01-16T15:01:26.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.471+0000 7f4f549fd740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f09a83dba0 0x55f09a86a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:26.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.472+0000 7f4f549fd740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55f09a8425e0 0x55f09a88fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:26.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.472+0000 7f4f549fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a842ea0 0x55f09a85aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:26.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.472+0000 7f4f549fd740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f09a7a9ac0 con 0x55f09a83dba0 2024-01-16T15:01:26.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.472+0000 7f4f549fd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f09a7462f0 con 0x55f09a842ea0 2024-01-16T15:01:26.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.472+0000 7f4f549fd740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f09a763c70 con 0x55f09a8425e0 2024-01-16T15:01:26.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.472+0000 7f4f549fd740 10 client.0 _ll_register_callbacks cb 0x55f09a7683b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-16T15:01:26.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.472+0000 7f4f4aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f09a83dba0 0x55f09a86a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:26.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.472+0000 7f4f4aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f09a83dba0 0x55f09a86a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40960/0 (socket says 172.21.15.143:40960) 2024-01-16T15:01:26.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.472+0000 7f4f4aa00640 1 -- 172.21.15.143:0/2673663152 learned_addr learned my addr 172.21.15.143:0/2673663152 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:26.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.472+0000 7f4f4aa00640 1 -- 172.21.15.143:0/2673663152 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55f09a8425e0 msgr2=0x55f09a88fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:26.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.472+0000 7f4f4aa00640 1 --2- 172.21.15.143:0/2673663152 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55f09a8425e0 0x55f09a88fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.472+0000 7f4f4aa00640 1 -- 172.21.15.143:0/2673663152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a842ea0 msgr2=0x55f09a85aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:26.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.472+0000 7f4f4aa00640 1 --2- 172.21.15.143:0/2673663152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a842ea0 0x55f09a85aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.472+0000 7f4f4aa00640 1 -- 172.21.15.143:0/2673663152 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f4c09c740 con 0x55f09a83dba0 2024-01-16T15:01:26.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.472+0000 7f4f4aa00640 1 --2- 172.21.15.143:0/2673663152 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f09a83dba0 0x55f09a86a730 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0766f0 tx=0x7f4f3c007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0f40d818977eea2 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:26.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.472+0000 7f4f33e00640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-01-16T15:01:26.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.473+0000 7f4f33e00640 1 -- 172.21.15.143:0/2673663152 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c07e160 con 0x55f09a83dba0 2024-01-16T15:01:26.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.473+0000 7f4f33e00640 1 -- 172.21.15.143:0/2673663152 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4f3c082580 con 0x55f09a83dba0 2024-01-16T15:01:26.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.473+0000 7f4f549fd740 1 -- 172.21.15.143:0/2673663152 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f09a7a2aa0 con 0x55f09a83dba0 2024-01-16T15:01:26.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.473+0000 7f4f33e00640 1 -- 172.21.15.143:0/2673663152 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f3c0796e0 con 0x55f09a83dba0 2024-01-16T15:01:26.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.473+0000 7f4f549fd740 10 client.5256 fetch_fsmap learned FSMap version 17 2024-01-16T15:01:26.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.473+0000 7f4f549fd740 1 -- 172.21.15.143:0/2673663152 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55f09a7a8140 con 0x55f09a83dba0 2024-01-16T15:01:26.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.473+0000 7f4f33e00640 1 -- 172.21.15.143:0/2673663152 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f09a7a2aa0 con 0x55f09a83dba0 2024-01-16T15:01:26.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.473+0000 7f4f33e00640 1 -- 172.21.15.143:0/2673663152 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4f3c04d020 con 0x55f09a83dba0 2024-01-16T15:01:26.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.473+0000 7f4f549fd740 10 client.5256 fetch_fsmap finished waiting for FSMap version 17 2024-01-16T15:01:26.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.473+0000 7f4f549fd740 10 client.5256 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-16T15:01:26.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.473+0000 7f4f549fd740 20 client.5256 populate_metadata read hostname 'smithi143' 2024-01-16T15:01:26.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.473+0000 7f4f549fd740 1 --2- 172.21.15.143:0/2673663152 >> [v2:172.21.15.143:6836/4104468169,v1:172.21.15.143:6837/4104468169] conn(0x55f09a869530 0x55f09a869910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:26.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.473+0000 7f4f549fd740 4 client.5256 mds_command: new command op to 5094 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-16T15:01:26.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.473+0000 7f4f549fd740 1 -- 172.21.15.143:0/2673663152 --> [v2:172.21.15.143:6836/4104468169,v1:172.21.15.143:6837/4104468169] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f09a7a27c0 con 0x55f09a869530 2024-01-16T15:01:26.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.474+0000 7f4f4a000640 1 --2- 172.21.15.143:0/2673663152 >> [v2:172.21.15.143:6836/4104468169,v1:172.21.15.143:6837/4104468169] conn(0x55f09a869530 0x55f09a869910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:26.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.474+0000 7f4f33e00640 10 client.5256 ms_handle_connect on v2:172.21.15.143:6836/4104468169 2024-01-16T15:01:26.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.474+0000 7f4f4a000640 1 --2- 172.21.15.143:0/2673663152 >> [v2:172.21.15.143:6836/4104468169,v1:172.21.15.143:6837/4104468169] conn(0x55f09a869530 0x55f09a869910 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=459ad4e059b176be server_cookie=2b525c132aed063a in_seq=0 out_seq=0 2024-01-16T15:01:26.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.475+0000 7f4f33e00640 1 -- 172.21.15.143:0/2673663152 <== mds.0 v2:172.21.15.143:6836/4104468169 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f09a7a27c0 con 0x55f09a869530 2024-01-16T15:01:26.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.475+0000 7f4f33e00640 10 client.5256 handle_command_reply: tid=0 2024-01-16T15:01:26.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.475+0000 7f4f549fd740 1 client.5256 shutdown 2024-01-16T15:01:26.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.475+0000 7f4f549fd740 1 -- 172.21.15.143:0/2673663152 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f09a83dba0 msgr2=0x55f09a86a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:26.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.475+0000 7f4f549fd740 1 --2- 172.21.15.143:0/2673663152 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f09a83dba0 0x55f09a86a730 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0766f0 tx=0x7f4f3c007c80 comp rx=0 tx=0).stop 2024-01-16T15:01:26.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.475+0000 7f4f549fd740 1 -- 172.21.15.143:0/2673663152 shutdown_connections 2024-01-16T15:01:26.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.475+0000 7f4f549fd740 1 --2- 172.21.15.143:0/2673663152 >> [v2:172.21.15.143:6836/4104468169,v1:172.21.15.143:6837/4104468169] conn(0x55f09a869530 0x55f09a869910 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.475+0000 7f4f549fd740 1 --2- 172.21.15.143:0/2673663152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a842ea0 0x55f09a85aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.475+0000 7f4f549fd740 1 --2- 172.21.15.143:0/2673663152 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55f09a8425e0 0x55f09a88fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.475+0000 7f4f33e00640 0 client.5256 ms_handle_reset on v2:172.21.15.143:6836/4104468169 2024-01-16T15:01:26.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.475+0000 7f4f549fd740 1 --2- 172.21.15.143:0/2673663152 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f09a83dba0 0x55f09a86a730 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.475+0000 7f4f549fd740 1 -- 172.21.15.143:0/2673663152 >> 172.21.15.143:0/2673663152 conn(0x55f09a804c70 msgr2=0x55f09a83cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:26.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.475+0000 7f4f549fd740 1 -- 172.21.15.143:0/2673663152 shutdown_connections 2024-01-16T15:01:26.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.476+0000 7f4f549fd740 1 -- 172.21.15.143:0/2673663152 wait complete. 2024-01-16T15:01:26.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.476+0000 7f4f549fd740 20 client.5256 trim_cache size 0 max 16384 2024-01-16T15:01:26.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.492+0000 7f4f549fd740 1 Processor -- start 2024-01-16T15:01:26.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.493+0000 7f4f549fd740 1 -- start start 2024-01-16T15:01:26.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.493+0000 7f4f549fd740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f09a83dba0 0x55f09a83cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:26.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.493+0000 7f4f549fd740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55f09a8425e0 0x55f09a83d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:26.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.493+0000 7f4f549fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a842ea0 0x55f09a870c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:26.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.493+0000 7f4f549fd740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f09a50cf20 con 0x55f09a83dba0 2024-01-16T15:01:26.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.493+0000 7f4f549fd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f09a8884d0 con 0x55f09a842ea0 2024-01-16T15:01:26.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.493+0000 7f4f549fd740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f09a88b3d0 con 0x55f09a8425e0 2024-01-16T15:01:26.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.494+0000 7f4f4aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f09a83dba0 0x55f09a83cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:26.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.494+0000 7f4f4aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f09a83dba0 0x55f09a83cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40968/0 (socket says 172.21.15.143:40968) 2024-01-16T15:01:26.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.494+0000 7f4f4aa00640 1 -- 172.21.15.143:0/2975396174 learned_addr learned my addr 172.21.15.143:0/2975396174 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:26.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.494+0000 7f4f4aa00640 1 -- 172.21.15.143:0/2975396174 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55f09a8425e0 msgr2=0x55f09a83d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:26.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.494+0000 7f4f4aa00640 1 --2- 172.21.15.143:0/2975396174 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55f09a8425e0 0x55f09a83d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.494+0000 7f4f4aa00640 1 -- 172.21.15.143:0/2975396174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a842ea0 msgr2=0x55f09a870c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:26.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.494+0000 7f4f4aa00640 1 --2- 172.21.15.143:0/2975396174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a842ea0 0x55f09a870c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.494+0000 7f4f4aa00640 1 -- 172.21.15.143:0/2975396174 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f09a745b80 con 0x55f09a83dba0 2024-01-16T15:01:26.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.494+0000 7f4f4aa00640 1 --2- 172.21.15.143:0/2975396174 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f09a83dba0 0x55f09a83cb80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c000c00 tx=0x7f4f3c0a7070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=902b377749e28e78 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:26.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.495+0000 7f4f3be00640 1 -- 172.21.15.143:0/2975396174 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c07e160 con 0x55f09a83dba0 2024-01-16T15:01:26.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.495+0000 7f4f3be00640 1 -- 172.21.15.143:0/2975396174 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f3c079b50 con 0x55f09a83dba0 2024-01-16T15:01:26.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.495+0000 7f4f3be00640 1 -- 172.21.15.143:0/2975396174 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c079930 con 0x55f09a83dba0 2024-01-16T15:01:26.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.495+0000 7f4f549fd740 1 -- 172.21.15.143:0/2975396174 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f09a83dba0 msgr2=0x55f09a83cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:26.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.495+0000 7f4f549fd740 1 --2- 172.21.15.143:0/2975396174 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f09a83dba0 0x55f09a83cb80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c000c00 tx=0x7f4f3c0a7070 comp rx=0 tx=0).stop 2024-01-16T15:01:26.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.495+0000 7f4f549fd740 1 -- 172.21.15.143:0/2975396174 shutdown_connections 2024-01-16T15:01:26.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.495+0000 7f4f549fd740 1 --2- 172.21.15.143:0/2975396174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a842ea0 0x55f09a870c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.495+0000 7f4f549fd740 1 --2- 172.21.15.143:0/2975396174 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55f09a8425e0 0x55f09a83d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.495+0000 7f4f549fd740 1 --2- 172.21.15.143:0/2975396174 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f09a83dba0 0x55f09a83cb80 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.495+0000 7f4f549fd740 1 -- 172.21.15.143:0/2975396174 >> 172.21.15.143:0/2975396174 conn(0x55f09a804c70 msgr2=0x55f09a88fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:26.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.495+0000 7f4f549fd740 1 -- 172.21.15.143:0/2975396174 shutdown_connections 2024-01-16T15:01:26.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.495+0000 7f4f549fd740 1 -- 172.21.15.143:0/2975396174 wait complete. 2024-01-16T15:01:26.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.496+0000 7f4f549fd740 1 -- start start 2024-01-16T15:01:26.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.496+0000 7f4f549fd740 1 Processor -- start 2024-01-16T15:01:26.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.497+0000 7f4f549fd740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f09a83dba0 0x55f09a88f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:26.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.497+0000 7f4f549fd740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55f09a8425e0 0x55f09a888990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:26.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.497+0000 7f4f549fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a842ea0 0x55f09a888ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:26.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.497+0000 7f4f549fd740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f09a86e8b0 con 0x55f09a83dba0 2024-01-16T15:01:26.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.497+0000 7f4f549fd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f09a843f30 con 0x55f09a842ea0 2024-01-16T15:01:26.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.497+0000 7f4f549fd740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f09a83d570 con 0x55f09a8425e0 2024-01-16T15:01:26.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.497+0000 7f4f549fd740 10 client.0 _ll_register_callbacks cb 0x7f4f3c079dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-16T15:01:26.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.497+0000 7f4f4aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f09a83dba0 0x55f09a88f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:26.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.497+0000 7f4f4aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f09a83dba0 0x55f09a88f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40982/0 (socket says 172.21.15.143:40982) 2024-01-16T15:01:26.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.497+0000 7f4f4aa00640 1 -- 172.21.15.143:0/4226845569 learned_addr learned my addr 172.21.15.143:0/4226845569 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:26.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.498+0000 7f4f4aa00640 1 -- 172.21.15.143:0/4226845569 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55f09a8425e0 msgr2=0x55f09a888990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:26.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.498+0000 7f4f4aa00640 1 --2- 172.21.15.143:0/4226845569 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55f09a8425e0 0x55f09a888990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.498+0000 7f4f4b400640 1 --2- 172.21.15.143:0/4226845569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a842ea0 0x55f09a888ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:26.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.498+0000 7f4f4aa00640 1 -- 172.21.15.143:0/4226845569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a842ea0 msgr2=0x55f09a888ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:26.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.498+0000 7f4f4aa00640 1 --2- 172.21.15.143:0/4226845569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a842ea0 0x55f09a888ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.498+0000 7f4f4aa00640 1 -- 172.21.15.143:0/4226845569 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f09a7a8140 con 0x55f09a83dba0 2024-01-16T15:01:26.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.498+0000 7f4f4aa00640 1 --2- 172.21.15.143:0/4226845569 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f09a83dba0 0x55f09a88f230 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c079ef0 tx=0x7f4f3c08e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf7542a5f9039b46 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:26.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.498+0000 7f4f33e00640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-01-16T15:01:26.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.498+0000 7f4f33e00640 1 -- 172.21.15.143:0/4226845569 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c07e160 con 0x55f09a83dba0 2024-01-16T15:01:26.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.498+0000 7f4f33e00640 1 -- 172.21.15.143:0/4226845569 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4f3c079930 con 0x55f09a83dba0 2024-01-16T15:01:26.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.498+0000 7f4f549fd740 1 -- 172.21.15.143:0/4226845569 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4f3c07e340 con 0x55f09a83dba0 2024-01-16T15:01:26.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.499+0000 7f4f33e00640 1 -- 172.21.15.143:0/4226845569 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f3c082580 con 0x55f09a83dba0 2024-01-16T15:01:26.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.499+0000 7f4f549fd740 10 client.5262 fetch_fsmap learned FSMap version 17 2024-01-16T15:01:26.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.499+0000 7f4f549fd740 1 -- 172.21.15.143:0/4226845569 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55f09a76b170 con 0x55f09a83dba0 2024-01-16T15:01:26.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.499+0000 7f4f33e00640 1 -- 172.21.15.143:0/4226845569 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f3c07e340 con 0x55f09a83dba0 2024-01-16T15:01:26.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.499+0000 7f4f33e00640 1 -- 172.21.15.143:0/4226845569 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4f3c0761f0 con 0x55f09a83dba0 2024-01-16T15:01:26.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.499+0000 7f4f549fd740 10 client.5262 fetch_fsmap finished waiting for FSMap version 17 2024-01-16T15:01:26.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.499+0000 7f4f549fd740 10 client.5262 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-16T15:01:26.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.499+0000 7f4f549fd740 20 client.5262 populate_metadata read hostname 'smithi143' 2024-01-16T15:01:26.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.499+0000 7f4f549fd740 1 --2- 172.21.15.143:0/4226845569 >> [v2:172.21.15.143:6836/4104468169,v1:172.21.15.143:6837/4104468169] conn(0x55f09a88fcf0 0x55f09a86db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:26.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.499+0000 7f4f549fd740 4 client.5262 mds_command: new command op to 5094 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-16T15:01:26.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.499+0000 7f4f549fd740 1 -- 172.21.15.143:0/4226845569 --> [v2:172.21.15.143:6836/4104468169,v1:172.21.15.143:6837/4104468169] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f09a4aa6c0 con 0x55f09a88fcf0 2024-01-16T15:01:26.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.499+0000 7f4f4a000640 1 --2- 172.21.15.143:0/4226845569 >> [v2:172.21.15.143:6836/4104468169,v1:172.21.15.143:6837/4104468169] conn(0x55f09a88fcf0 0x55f09a86db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:26.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.500+0000 7f4f4a000640 1 --2- 172.21.15.143:0/4226845569 >> [v2:172.21.15.143:6836/4104468169,v1:172.21.15.143:6837/4104468169] conn(0x55f09a88fcf0 0x55f09a86db40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3324f2eab0fcef0b server_cookie=c1b56c3cd4d7962c in_seq=0 out_seq=0 2024-01-16T15:01:26.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.500+0000 7f4f33e00640 10 client.5262 ms_handle_connect on v2:172.21.15.143:6836/4104468169 2024-01-16T15:01:26.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.500+0000 7f4f33e00640 1 -- 172.21.15.143:0/4226845569 <== mds.0 v2:172.21.15.143:6836/4104468169 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55f09a4aa6c0 con 0x55f09a88fcf0 2024-01-16T15:01:26.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.500+0000 7f4f33e00640 10 client.5262 handle_command_reply: tid=0 2024-01-16T15:01:26.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.500+0000 7f4f549fd740 1 client.5262 shutdown 2024-01-16T15:01:26.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.500+0000 7f4f549fd740 1 -- 172.21.15.143:0/4226845569 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f09a83dba0 msgr2=0x55f09a88f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:26.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.500+0000 7f4f549fd740 1 --2- 172.21.15.143:0/4226845569 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f09a83dba0 0x55f09a88f230 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c079ef0 tx=0x7f4f3c08e000 comp rx=0 tx=0).stop 2024-01-16T15:01:26.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.500+0000 7f4f549fd740 1 -- 172.21.15.143:0/4226845569 shutdown_connections 2024-01-16T15:01:26.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.500+0000 7f4f549fd740 1 --2- 172.21.15.143:0/4226845569 >> [v2:172.21.15.143:6836/4104468169,v1:172.21.15.143:6837/4104468169] conn(0x55f09a88fcf0 0x55f09a86db40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.500+0000 7f4f549fd740 1 --2- 172.21.15.143:0/4226845569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09a842ea0 0x55f09a888ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.500+0000 7f4f549fd740 1 --2- 172.21.15.143:0/4226845569 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55f09a8425e0 0x55f09a888990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.500+0000 7f4f33e00640 0 client.5262 ms_handle_reset on v2:172.21.15.143:6836/4104468169 2024-01-16T15:01:26.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.500+0000 7f4f549fd740 1 --2- 172.21.15.143:0/4226845569 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f09a83dba0 0x55f09a88f230 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.500+0000 7f4f549fd740 1 -- 172.21.15.143:0/4226845569 >> 172.21.15.143:0/4226845569 conn(0x55f09a804c70 msgr2=0x55f09a83cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:26.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.500+0000 7f4f549fd740 1 -- 172.21.15.143:0/4226845569 shutdown_connections 2024-01-16T15:01:26.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.500+0000 7f4f549fd740 1 -- 172.21.15.143:0/4226845569 wait complete. 2024-01-16T15:01:26.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.501+0000 7f4f549fd740 20 client.5262 trim_cache size 0 max 16384 2024-01-16T15:01:26.484 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-16T15:01:26.484 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0", 2024-01-16T15:01:26.484 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 0, 2024-01-16T15:01:26.485 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 5094, 2024-01-16T15:01:26.485 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2024-01-16T15:01:26.485 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2024-01-16T15:01:26.485 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2024-01-16T15:01:26.485 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 6.6175037120000004, 2024-01-16T15:01:26.485 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 17, 2024-01-16T15:01:26.485 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 34, 2024-01-16T15:01:26.485 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 34, 2024-01-16T15:01:26.485 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 16.474382201000001 2024-01-16T15:01:26.485 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-16T15:01:26.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.501+0000 7f4f51c00640 1 -- 172.21.15.143:0/3142647184 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f4f2406b2f0 msgr2=0x7f4f2406d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:26.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.501+0000 7f4f51c00640 1 --2- 172.21.15.143:0/3142647184 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f4f2406b2f0 0x7f4f2406d7b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4f340018e0 tx=0x7f4f34009040 comp rx=0 tx=0).stop 2024-01-16T15:01:26.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.501+0000 7f4f51c00640 1 -- 172.21.15.143:0/3142647184 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f2c003970 msgr2=0x7f4f4c1ee4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:26.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.501+0000 7f4f51c00640 1 --2- 172.21.15.143:0/3142647184 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f2c003970 0x7f4f4c1ee4d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c02bc30 tx=0x7f4f3c001120 comp rx=0 tx=0).stop 2024-01-16T15:01:26.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.501+0000 7f4f51c00640 1 -- 172.21.15.143:0/3142647184 shutdown_connections 2024-01-16T15:01:26.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.501+0000 7f4f51c00640 1 --2- 172.21.15.143:0/3142647184 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f4f2406b2f0 0x7f4f2406d7b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.501+0000 7f4f51c00640 1 --2- 172.21.15.143:0/3142647184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f4c146e70 0x7f4f4c147670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.501+0000 7f4f51c00640 1 --2- 172.21.15.143:0/3142647184 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f4c146510 0x7f4f4c146930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.501+0000 7f4f51c00640 1 --2- 172.21.15.143:0/3142647184 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f2c003970 0x7f4f4c1ee4d0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.501+0000 7f4f51c00640 1 -- 172.21.15.143:0/3142647184 >> 172.21.15.143:0/3142647184 conn(0x7f4f4c0b0c50 msgr2=0x7f4f4c0ccb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:26.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.501+0000 7f4f51c00640 1 -- 172.21.15.143:0/3142647184 shutdown_connections 2024-01-16T15:01:26.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.501+0000 7f4f51c00640 1 -- 172.21.15.143:0/3142647184 wait complete. 2024-01-16T15:01:26.498 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-16T15:01:26.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.719+0000 7fb199800640 1 Processor -- start 2024-01-16T15:01:26.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.719+0000 7fb199800640 1 -- start start 2024-01-16T15:01:26.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.721+0000 7fb199800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb194144b30 0x7fb19415f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:26.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.721+0000 7fb199800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb19415c410 0x7fb1940b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:26.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.721+0000 7fb199800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb19415bf80 con 0x7fb1940b1de0 2024-01-16T15:01:26.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.721+0000 7fb199800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1940b1560 con 0x7fb194144b30 2024-01-16T15:01:26.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.721+0000 7fb199800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1940b0fb0 con 0x7fb19415c410 2024-01-16T15:01:26.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.721+0000 7fb198e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb1940b1de0 0x7fb19406c5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:58500/0 (socket says 172.21.15.143:58500) 2024-01-16T15:01:26.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.721+0000 7fb198e00640 1 -- 172.21.15.143:0/170505173 learned_addr learned my addr 172.21.15.143:0/170505173 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:01:26.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.721+0000 7fb193400640 1 --2- 172.21.15.143:0/170505173 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb19415c410 0x7fb1940b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:26.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.721+0000 7fb193e00640 1 --2- 172.21.15.143:0/170505173 >> v2:172.21.15.204:3300/0 conn(0x7fb194144b30 0x7fb19415f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:26.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.722+0000 7fb192a00640 1 -- 172.21.15.143:0/170505173 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3621486304 0 0) 0x7fb19415bf80 con 0x7fb1940b1de0 2024-01-16T15:01:26.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.722+0000 7fb192a00640 1 -- 172.21.15.143:0/170505173 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb174003520 con 0x7fb1940b1de0 2024-01-16T15:01:26.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.722+0000 7fb192a00640 1 -- 172.21.15.143:0/170505173 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1590523508 0 0) 0x7fb174003520 con 0x7fb1940b1de0 2024-01-16T15:01:26.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.722+0000 7fb192a00640 1 -- 172.21.15.143:0/170505173 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb19415c410 msgr2=0x7fb1940b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:26.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.722+0000 7fb192a00640 1 --2- 172.21.15.143:0/170505173 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb19415c410 0x7fb1940b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.722+0000 7fb192a00640 1 -- 172.21.15.143:0/170505173 >> v2:172.21.15.204:3300/0 conn(0x7fb194144b30 msgr2=0x7fb19415f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:26.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.722+0000 7fb192a00640 1 --2- 172.21.15.143:0/170505173 >> v2:172.21.15.204:3300/0 conn(0x7fb194144b30 0x7fb19415f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.722+0000 7fb192a00640 1 -- 172.21.15.143:0/170505173 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1940b1280 con 0x7fb1940b1de0 2024-01-16T15:01:26.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.722+0000 7fb192a00640 1 -- 172.21.15.143:0/170505173 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7fb18c0019f0 con 0x7fb1940b1de0 2024-01-16T15:01:26.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.722+0000 7fb192a00640 1 -- 172.21.15.143:0/170505173 >> v1:172.21.15.143:6789/0 conn(0x7fb1940b1de0 legacy=0x7fb19406c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:26.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.722+0000 7fb192a00640 1 --2- 172.21.15.143:0/170505173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb174003970 0x7fb174003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:26.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.722+0000 7fb192a00640 1 --2- 172.21.15.143:0/170505173 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb19415c410 0x7fb1740047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:26.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.722+0000 7fb192a00640 1 --2- 172.21.15.143:0/170505173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb174004d00 0x7fb174005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:26.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.722+0000 7fb192a00640 1 -- 172.21.15.143:0/170505173 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1940b1560 con 0x7fb174004d00 2024-01-16T15:01:26.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.722+0000 7fb192a00640 1 -- 172.21.15.143:0/170505173 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1940b0fb0 con 0x7fb174003970 2024-01-16T15:01:26.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.722+0000 7fb192a00640 1 -- 172.21.15.143:0/170505173 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb17400b980 con 0x7fb19415c410 2024-01-16T15:01:26.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.722+0000 7fb198e00640 1 --2- 172.21.15.143:0/170505173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb174004d00 0x7fb174005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:26.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.723+0000 7fb193400640 1 --2- 172.21.15.143:0/170505173 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb19415c410 0x7fb1740047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:26.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.723+0000 7fb198e00640 1 -- 172.21.15.143:0/170505173 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb19415c410 msgr2=0x7fb1740047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:26.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.723+0000 7fb198e00640 1 --2- 172.21.15.143:0/170505173 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb19415c410 0x7fb1740047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.723+0000 7fb198e00640 1 -- 172.21.15.143:0/170505173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb174003970 msgr2=0x7fb174003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:26.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.723+0000 7fb198e00640 1 --2- 172.21.15.143:0/170505173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb174003970 0x7fb174003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.723+0000 7fb198e00640 1 -- 172.21.15.143:0/170505173 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb174003520 con 0x7fb174004d00 2024-01-16T15:01:26.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.724+0000 7fb198e00640 1 --2- 172.21.15.143:0/170505173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb174004d00 0x7fb174005180 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00a860 tx=0x7fb1740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda379873874eb81 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:26.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.724+0000 7fb192a00640 1 -- 172.21.15.143:0/170505173 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18c034630 con 0x7fb174004d00 2024-01-16T15:01:26.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.724+0000 7fb192a00640 1 -- 172.21.15.143:0/170505173 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb18c02f070 con 0x7fb174004d00 2024-01-16T15:01:26.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.724+0000 7fb192a00640 1 -- 172.21.15.143:0/170505173 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18c034a70 con 0x7fb174004d00 2024-01-16T15:01:26.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.725+0000 7fb199800640 1 -- 172.21.15.143:0/170505173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb174004d00 msgr2=0x7fb174005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:26.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.725+0000 7fb199800640 1 --2- 172.21.15.143:0/170505173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb174004d00 0x7fb174005180 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00a860 tx=0x7fb1740057f0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.725+0000 7fb198e00640 1 -- 172.21.15.143:0/170505173 reap_dead start 2024-01-16T15:01:26.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.725+0000 7fb199800640 1 -- 172.21.15.143:0/170505173 shutdown_connections 2024-01-16T15:01:26.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.725+0000 7fb199800640 1 -- 172.21.15.143:0/170505173 >> 172.21.15.143:0/170505173 conn(0x7fb1940cb830 msgr2=0x7fb1940cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:26.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.725+0000 7fb199800640 1 -- 172.21.15.143:0/170505173 shutdown_connections 2024-01-16T15:01:26.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.725+0000 7fb199800640 1 -- 172.21.15.143:0/170505173 wait complete. 2024-01-16T15:01:26.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.727+0000 7fb199800640 1 Processor -- start 2024-01-16T15:01:26.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.727+0000 7fb199800640 1 -- start start 2024-01-16T15:01:26.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.728+0000 7fb199800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb174004d00 0x7fb194157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:26.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.728+0000 7fb199800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb194137860 0x7fb1941f09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:26.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.728+0000 7fb199800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1941f4800 0x7fb194137e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:26.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.728+0000 7fb199800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1941f4020 con 0x7fb194137860 2024-01-16T15:01:26.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.728+0000 7fb199800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1941f1310 con 0x7fb1941f4800 2024-01-16T15:01:26.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.728+0000 7fb199800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1941f15f0 con 0x7fb174004d00 2024-01-16T15:01:26.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.728+0000 7fb193400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb194137860 0x7fb1941f09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:26.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.728+0000 7fb193400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb194137860 0x7fb1941f09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41004/0 (socket says 172.21.15.143:41004) 2024-01-16T15:01:26.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.728+0000 7fb193400640 1 -- 172.21.15.143:0/1962303810 learned_addr learned my addr 172.21.15.143:0/1962303810 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:26.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.728+0000 7fb193400640 1 -- 172.21.15.143:0/1962303810 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb174004d00 msgr2=0x7fb194157030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:26.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.728+0000 7fb193400640 1 --2- 172.21.15.143:0/1962303810 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb174004d00 0x7fb194157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.728+0000 7fb193400640 1 -- 172.21.15.143:0/1962303810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1941f4800 msgr2=0x7fb194137e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:26.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.728+0000 7fb193400640 1 --2- 172.21.15.143:0/1962303810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1941f4800 0x7fb194137e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.728+0000 7fb193400640 1 -- 172.21.15.143:0/1962303810 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1941f18d0 con 0x7fb194137860 2024-01-16T15:01:26.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.729+0000 7fb193400640 1 --2- 172.21.15.143:0/1962303810 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb194137860 0x7fb1941f09f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb17c010860 tx=0x7fb17c010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:26.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.729+0000 7fb18be00640 1 -- 172.21.15.143:0/1962303810 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17c009610 con 0x7fb194137860 2024-01-16T15:01:26.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.729+0000 7fb18be00640 1 -- 172.21.15.143:0/1962303810 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb17c00eae0 con 0x7fb194137860 2024-01-16T15:01:26.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.729+0000 7fb18be00640 1 -- 172.21.15.143:0/1962303810 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17c00ee70 con 0x7fb194137860 2024-01-16T15:01:26.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.729+0000 7fb199800640 1 -- 172.21.15.143:0/1962303810 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1941f42e0 con 0x7fb194137860 2024-01-16T15:01:26.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.729+0000 7fb199800640 1 -- 172.21.15.143:0/1962303810 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1941f4c80 con 0x7fb194137860 2024-01-16T15:01:26.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.730+0000 7fb199800640 1 -- 172.21.15.143:0/1962303810 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb19404e840 con 0x7fb194137860 2024-01-16T15:01:26.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.731+0000 7fb18be00640 1 -- 172.21.15.143:0/1962303810 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fb17c01c400 con 0x7fb194137860 2024-01-16T15:01:26.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.733+0000 7fb18be00640 1 --2- 172.21.15.143:0/1962303810 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb16c06b2c0 0x7fb16c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:26.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.733+0000 7fb18be00640 1 -- 172.21.15.143:0/1962303810 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb17c08a130 con 0x7fb194137860 2024-01-16T15:01:26.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.733+0000 7fb193e00640 1 --2- 172.21.15.143:0/1962303810 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb16c06b2c0 0x7fb16c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:26.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.733+0000 7fb193e00640 1 --2- 172.21.15.143:0/1962303810 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb16c06b2c0 0x7fb16c06d780 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb184009a20 tx=0x7fb1840094e0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:26.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.737+0000 7fb18be00640 1 -- 172.21.15.143:0/1962303810 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb17c055040 con 0x7fb194137860 2024-01-16T15:01:26.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.970+0000 7fb199800640 1 -- 172.21.15.143:0/1962303810 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb19406d860 con 0x7fb194137860 2024-01-16T15:01:26.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.971+0000 7fb18be00640 1 -- 172.21.15.143:0/1962303810 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb17c058660 con 0x7fb194137860 2024-01-16T15:01:26.952 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-16T15:01:26.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.973+0000 7fb199800640 1 -- 172.21.15.143:0/1962303810 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb16c06b2c0 msgr2=0x7fb16c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:26.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.973+0000 7fb199800640 1 --2- 172.21.15.143:0/1962303810 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb16c06b2c0 0x7fb16c06d780 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb184009a20 tx=0x7fb1840094e0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.973+0000 7fb199800640 1 -- 172.21.15.143:0/1962303810 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb194137860 msgr2=0x7fb1941f09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:26.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.973+0000 7fb199800640 1 --2- 172.21.15.143:0/1962303810 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb194137860 0x7fb1941f09f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb17c010860 tx=0x7fb17c010d20 comp rx=0 tx=0).stop 2024-01-16T15:01:26.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.974+0000 7fb199800640 1 -- 172.21.15.143:0/1962303810 shutdown_connections 2024-01-16T15:01:26.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.974+0000 7fb199800640 1 --2- 172.21.15.143:0/1962303810 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb16c06b2c0 0x7fb16c06d780 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.974+0000 7fb199800640 1 --2- 172.21.15.143:0/1962303810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1941f4800 0x7fb194137e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.974+0000 7fb199800640 1 --2- 172.21.15.143:0/1962303810 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb194137860 0x7fb1941f09f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.974+0000 7fb199800640 1 --2- 172.21.15.143:0/1962303810 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb174004d00 0x7fb194157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:26.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.974+0000 7fb199800640 1 -- 172.21.15.143:0/1962303810 >> 172.21.15.143:0/1962303810 conn(0x7fb1940cb830 msgr2=0x7fb194143f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:26.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.974+0000 7fb199800640 1 -- 172.21.15.143:0/1962303810 shutdown_connections 2024-01-16T15:01:26.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:26.974+0000 7fb199800640 1 -- 172.21.15.143:0/1962303810 wait complete. 2024-01-16T15:01:26.973 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-16T15:01:26.973 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-16T15:01:26.973 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi143.front.sepia.ceph.com 2024-01-16T15:01:26.973 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-16T15:01:26.973 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-16T15:01:26.974 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-16T15:01:26.974 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-16T15:01:26.974 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-16T15:01:26.974 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-16T15:01:26.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:01:26.974 DEBUG:teuthology.orchestra.run.smithi143:> ip addr 2024-01-16T15:01:27.023 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-16T15:01:27.023 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-16T15:01:27.023 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2024-01-16T15:01:27.023 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:01:27.023 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2024-01-16T15:01:27.023 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:01:27.023 INFO:teuthology.orchestra.run.smithi143.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-16T15:01:27.023 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:94 brd ff:ff:ff:ff:ff:ff 2024-01-16T15:01:27.023 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f0 2024-01-16T15:01:27.023 INFO:teuthology.orchestra.run.smithi143.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-16T15:01:27.023 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:95 brd ff:ff:ff:ff:ff:ff 2024-01-16T15:01:27.023 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f1 2024-01-16T15:01:27.023 INFO:teuthology.orchestra.run.smithi143.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-16T15:01:27.024 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:82 brd ff:ff:ff:ff:ff:ff 2024-01-16T15:01:27.024 INFO:teuthology.orchestra.run.smithi143.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-16T15:01:27.024 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:83 brd ff:ff:ff:ff:ff:ff 2024-01-16T15:01:27.024 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.21.15.143/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-16T15:01:27.024 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft 42530sec preferred_lft 42530sec 2024-01-16T15:01:27.024 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::ec4:7aff:fe88:7783/64 scope link 2024-01-16T15:01:27.024 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:01:27.024 INFO:teuthology.orchestra.run.smithi143.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-16T15:01:27.024 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether ce:00:1c:52:30:ff brd ff:ff:ff:ff:ff:ff 2024-01-16T15:01:27.024 INFO:teuthology.orchestra.run.smithi143.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-16T15:01:27.024 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:01:27.024 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::d08e:67ff:fe88:3a91/64 scope link 2024-01-16T15:01:27.024 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:01:27.024 INFO:teuthology.orchestra.run.smithi143.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-16T15:01:27.024 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether ce:00:1c:52:30:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-16T15:01:27.025 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::cc00:1cff:fe52:30ff/64 scope link 2024-01-16T15:01:27.025 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:01:27.025 INFO:teuthology.orchestra.run.smithi143.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-16T15:01:27.025 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether de:f2:e1:77:0c:bc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-16T15:01:27.025 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::dcf2:e1ff:fe77:cbc/64 scope link 2024-01-16T15:01:27.025 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:01:27.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:01:27.026 DEBUG:teuthology.orchestra.run.smithi143:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-16T15:01:27.114 INFO:teuthology.orchestra.run.smithi143.stdout:1 2024-01-16T15:01:27.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:01:27.119 DEBUG:teuthology.orchestra.run.smithi143:> route 2024-01-16T15:01:27.172 INFO:teuthology.orchestra.run.smithi143.stdout:Kernel IP routing table 2024-01-16T15:01:27.172 INFO:teuthology.orchestra.run.smithi143.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-16T15:01:27.172 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-16T15:01:27.172 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-16T15:01:27.172 INFO:teuthology.orchestra.run.smithi143.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-16T15:01:27.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:01:27.173 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2024-01-16T15:01:27.173 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-01-16T15:01:27.173 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-16T15:01:27.173 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-16T15:01:27.173 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-16T15:01:27.173 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-01-16T15:01:27.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:01:27.355 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2024-01-16T15:01:27.404 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-16T15:01:27.405 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-16T15:01:27.405 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-16T15:01:27.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:01:27.406 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set brx.0 up 2024-01-16T15:01:27.501 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-16T15:01:27.501 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-16T15:01:27.554 INFO:teuthology.orchestra.run.smithi143.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-16T15:01:27.555 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-16T15:01:27.555 DEBUG:teuthology.orchestra.run.smithi143:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-16T15:01:27.604 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-16T15:01:27.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:01:27.604 DEBUG:teuthology.orchestra.run.smithi143:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-16T15:01:27.742 INFO:teuthology.orchestra.run.smithi143.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-16T15:01:27.742 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-16T15:01:27.742 INFO:teuthology.orchestra.run.smithi143.stdout:invalid new device string format 2024-01-16T15:01:27.764 INFO:teuthology.orchestra.run.smithi143.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-16T15:01:27.764 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-16T15:01:27.764 INFO:teuthology.orchestra.run.smithi143.stdout:invalid new device string format 2024-01-16T15:01:27.764 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: resolved to: "172.21.15.143:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-01-16T15:01:27.764 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-16T15:01:27.791 INFO:teuthology.orchestra.run.smithi143.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-16T15:01:27.791 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-16T15:01:27.791 INFO:teuthology.orchestra.run.smithi143.stdout:invalid new device string format 2024-01-16T15:01:27.791 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: resolved to: "172.21.15.143:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-01-16T15:01:27.791 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-16T15:01:27.791 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: resolved to: "172.21.15.143:6789,172.21.15.204:6789,172.21.15.204:6790" 2024-01-16T15:01:27.791 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.143:6789,172.21.15.204:6789,172.21.15.204:6790:/ 2024-01-16T15:01:27.791 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0" will pass to kernel 2024-01-16T15:01:27.793 INFO:teuthology.orchestra.run.smithi143.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-01-16T15:01:27.793 INFO:teuthology.orchestra.run.smithi143.stdout: You just mounted a file system that supports labels which does not 2024-01-16T15:01:27.793 INFO:teuthology.orchestra.run.smithi143.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-01-16T15:01:27.793 INFO:teuthology.orchestra.run.smithi143.stdout: applications will generate AVC messages and not be allowed access to 2024-01-16T15:01:27.793 INFO:teuthology.orchestra.run.smithi143.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-01-16T15:01:27.798 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-16T15:01:27.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:01:27.799 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-16T15:01:27.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:01:27.876 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-16T15:01:27.935 INFO:teuthology.orchestra.run.smithi143.stdout:client4655 2024-01-16T15:01:27.936 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T15:01:27.936 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T15:01:27.981 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-16T15:01:28.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.254+0000 7fd2dd800640 1 Processor -- start 2024-01-16T15:01:28.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.254+0000 7fd2dd800640 1 -- start start 2024-01-16T15:01:28.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.256+0000 7fd2dd800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd2d8157c70 0x7fd2d8161df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:28.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.256+0000 7fd2dd800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2d815e390 0x7fd2d80cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:28.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.256+0000 7fd2dd800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2d80ce0b0 con 0x7fd2d80cf450 2024-01-16T15:01:28.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.256+0000 7fd2dd800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2d80cfe40 con 0x7fd2d8157c70 2024-01-16T15:01:28.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.256+0000 7fd2dd800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d8161960 con 0x7fd2d815e390 2024-01-16T15:01:28.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.256+0000 7fd2d7400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2d815e390 0x7fd2d80cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:28.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.256+0000 7fd2d6a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd2d8157c70 0x7fd2d8161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:28.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.256+0000 7fd2d7400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2d815e390 0x7fd2d80cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:46700/0 (socket says 172.21.15.143:46700) 2024-01-16T15:01:28.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.256+0000 7fd2d7400640 1 -- 172.21.15.143:0/3359653752 learned_addr learned my addr 172.21.15.143:0/3359653752 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:28.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.257+0000 7fd2d6a00640 1 -- 172.21.15.143:0/3359653752 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2d815e390 msgr2=0x7fd2d80cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:28.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.257+0000 7fd2d6a00640 1 --2- 172.21.15.143:0/3359653752 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2d815e390 0x7fd2d80cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:28.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.257+0000 7fd2d6a00640 1 -- 172.21.15.143:0/3359653752 >> v1:172.21.15.204:6789/0 conn(0x7fd2d80cf450 legacy=0x7fd2d815dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:28.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.257+0000 7fd2d6a00640 1 -- 172.21.15.143:0/3359653752 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d815e8d0 con 0x7fd2d8157c70 2024-01-16T15:01:28.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.259+0000 7fd2d6a00640 1 --2- 172.21.15.143:0/3359653752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2d8157c70 0x7fd2d8161df0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd2c80097a0 tx=0x7fd2c802b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aca1a2c19d04ceb3 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:28.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.259+0000 7fd2d6000640 1 -- 172.21.15.143:0/3359653752 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c8031780 con 0x7fd2d8157c70 2024-01-16T15:01:28.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.259+0000 7fd2d6000640 1 -- 172.21.15.143:0/3359653752 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c80318e0 con 0x7fd2d8157c70 2024-01-16T15:01:28.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.259+0000 7fd2d6000640 1 -- 172.21.15.143:0/3359653752 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c8031c70 con 0x7fd2d8157c70 2024-01-16T15:01:28.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.260+0000 7fd2dd800640 1 -- 172.21.15.143:0/3359653752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2d8157c70 msgr2=0x7fd2d8161df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:28.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.260+0000 7fd2dd800640 1 --2- 172.21.15.143:0/3359653752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2d8157c70 0x7fd2d8161df0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd2c80097a0 tx=0x7fd2c802b820 comp rx=0 tx=0).stop 2024-01-16T15:01:28.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.260+0000 7fd2dd800640 1 -- 172.21.15.143:0/3359653752 shutdown_connections 2024-01-16T15:01:28.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.260+0000 7fd2dd800640 1 --2- 172.21.15.143:0/3359653752 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2d815e390 0x7fd2d80cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:28.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.260+0000 7fd2dd800640 1 --2- 172.21.15.143:0/3359653752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2d8157c70 0x7fd2d8161df0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:28.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.260+0000 7fd2dd800640 1 -- 172.21.15.143:0/3359653752 >> 172.21.15.143:0/3359653752 conn(0x7fd2d806a920 msgr2=0x7fd2d806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:28.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.260+0000 7fd2dd800640 1 -- 172.21.15.143:0/3359653752 shutdown_connections 2024-01-16T15:01:28.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.260+0000 7fd2dd800640 1 -- 172.21.15.143:0/3359653752 wait complete. 2024-01-16T15:01:28.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.262+0000 7fd2dd800640 1 Processor -- start 2024-01-16T15:01:28.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.262+0000 7fd2dd800640 1 -- start start 2024-01-16T15:01:28.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.263+0000 7fd2dd800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2d80cf450 0x7fd2d80ae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:28.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.263+0000 7fd2dd800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2d8157c70 0x7fd2d8099b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:28.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.263+0000 7fd2dd800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2d815e390 0x7fd2d815f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:28.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.263+0000 7fd2dd800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d809a090 con 0x7fd2d80cf450 2024-01-16T15:01:28.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.263+0000 7fd2dd800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d80aea80 con 0x7fd2d8157c70 2024-01-16T15:01:28.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.263+0000 7fd2dd800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d80aed60 con 0x7fd2d815e390 2024-01-16T15:01:28.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.263+0000 7fd2d6a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2d80cf450 0x7fd2d80ae3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:28.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.263+0000 7fd2d6a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2d80cf450 0x7fd2d80ae3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41012/0 (socket says 172.21.15.143:41012) 2024-01-16T15:01:28.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.263+0000 7fd2d6a00640 1 -- 172.21.15.143:0/2252569940 learned_addr learned my addr 172.21.15.143:0/2252569940 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:28.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.264+0000 7fd2d7400640 1 --2- 172.21.15.143:0/2252569940 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2d815e390 0x7fd2d815f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:28.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.264+0000 7fd2cfe00640 1 --2- 172.21.15.143:0/2252569940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2d8157c70 0x7fd2d8099b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:28.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.264+0000 7fd2d6a00640 1 -- 172.21.15.143:0/2252569940 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2d815e390 msgr2=0x7fd2d815f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:28.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.264+0000 7fd2d6a00640 1 --2- 172.21.15.143:0/2252569940 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2d815e390 0x7fd2d815f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:28.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.264+0000 7fd2d6a00640 1 -- 172.21.15.143:0/2252569940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2d8157c70 msgr2=0x7fd2d8099b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:28.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.264+0000 7fd2d6a00640 1 --2- 172.21.15.143:0/2252569940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2d8157c70 0x7fd2d8099b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:28.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.264+0000 7fd2d6a00640 1 -- 172.21.15.143:0/2252569940 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c8009400 con 0x7fd2d80cf450 2024-01-16T15:01:28.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.264+0000 7fd2d7400640 1 --2- 172.21.15.143:0/2252569940 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2d815e390 0x7fd2d815f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T15:01:28.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.264+0000 7fd2d6a00640 1 --2- 172.21.15.143:0/2252569940 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2d80cf450 0x7fd2d80ae3a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8034040 tx=0x7fd2c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:28.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.265+0000 7fd2cea00640 1 -- 172.21.15.143:0/2252569940 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c8032820 con 0x7fd2d80cf450 2024-01-16T15:01:28.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.265+0000 7fd2cea00640 1 -- 172.21.15.143:0/2252569940 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c8032980 con 0x7fd2d80cf450 2024-01-16T15:01:28.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.265+0000 7fd2cea00640 1 -- 172.21.15.143:0/2252569940 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c8032d30 con 0x7fd2d80cf450 2024-01-16T15:01:28.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.265+0000 7fd2dd800640 1 -- 172.21.15.143:0/2252569940 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2d80af040 con 0x7fd2d80cf450 2024-01-16T15:01:28.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.265+0000 7fd2dd800640 1 -- 172.21.15.143:0/2252569940 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d80af600 con 0x7fd2d80cf450 2024-01-16T15:01:28.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.266+0000 7fd2dd800640 1 -- 172.21.15.143:0/2252569940 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2d8067470 con 0x7fd2d80cf450 2024-01-16T15:01:28.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.267+0000 7fd2cea00640 1 -- 172.21.15.143:0/2252569940 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fd2c804e440 con 0x7fd2d80cf450 2024-01-16T15:01:28.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.268+0000 7fd2cea00640 1 --2- 172.21.15.143:0/2252569940 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd2b006b2c0 0x7fd2b006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:28.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.268+0000 7fd2cea00640 1 -- 172.21.15.143:0/2252569940 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd2c80abd50 con 0x7fd2d80cf450 2024-01-16T15:01:28.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.268+0000 7fd2cfe00640 1 --2- 172.21.15.143:0/2252569940 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd2b006b2c0 0x7fd2b006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:28.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.269+0000 7fd2cfe00640 1 --2- 172.21.15.143:0/2252569940 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd2b006b2c0 0x7fd2b006d780 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd2d806de50 tx=0x7fd2c0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:28.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.272+0000 7fd2cea00640 1 -- 172.21.15.143:0/2252569940 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd2c8076b20 con 0x7fd2d80cf450 2024-01-16T15:01:28.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.500+0000 7fd2dd800640 1 -- 172.21.15.143:0/2252569940 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd2d80cd910 con 0x7fd2d80cf450 2024-01-16T15:01:28.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.501+0000 7fd2cea00640 1 -- 172.21.15.143:0/2252569940 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd2c807a140 con 0x7fd2d80cf450 2024-01-16T15:01:28.482 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-16T15:01:28.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.504+0000 7fd2dd800640 1 -- 172.21.15.143:0/2252569940 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd2b006b2c0 msgr2=0x7fd2b006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:28.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.504+0000 7fd2dd800640 1 --2- 172.21.15.143:0/2252569940 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd2b006b2c0 0x7fd2b006d780 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd2d806de50 tx=0x7fd2c0009040 comp rx=0 tx=0).stop 2024-01-16T15:01:28.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.504+0000 7fd2dd800640 1 -- 172.21.15.143:0/2252569940 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2d80cf450 msgr2=0x7fd2d80ae3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:28.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.504+0000 7fd2dd800640 1 --2- 172.21.15.143:0/2252569940 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2d80cf450 0x7fd2d80ae3a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8034040 tx=0x7fd2c80046c0 comp rx=0 tx=0).stop 2024-01-16T15:01:28.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.504+0000 7fd2dd800640 1 -- 172.21.15.143:0/2252569940 shutdown_connections 2024-01-16T15:01:28.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.504+0000 7fd2dd800640 1 --2- 172.21.15.143:0/2252569940 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd2b006b2c0 0x7fd2b006d780 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:28.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.504+0000 7fd2dd800640 1 --2- 172.21.15.143:0/2252569940 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2d815e390 0x7fd2d815f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:28.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.504+0000 7fd2dd800640 1 --2- 172.21.15.143:0/2252569940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2d8157c70 0x7fd2d8099b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:28.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.504+0000 7fd2dd800640 1 --2- 172.21.15.143:0/2252569940 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2d80cf450 0x7fd2d80ae3a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:28.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.504+0000 7fd2dd800640 1 -- 172.21.15.143:0/2252569940 >> 172.21.15.143:0/2252569940 conn(0x7fd2d806a920 msgr2=0x7fd2d80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:28.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.504+0000 7fd2dd800640 1 -- 172.21.15.143:0/2252569940 shutdown_connections 2024-01-16T15:01:28.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:28.504+0000 7fd2dd800640 1 -- 172.21.15.143:0/2252569940 wait complete. 2024-01-16T15:01:28.503 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-16T15:01:28.504 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-16T15:01:28.504 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi204.front.sepia.ceph.com 2024-01-16T15:01:28.504 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-16T15:01:28.504 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-16T15:01:28.504 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-16T15:01:28.504 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-16T15:01:28.504 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-16T15:01:28.504 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-16T15:01:28.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:01:28.505 DEBUG:teuthology.orchestra.run.smithi204:> ip addr 2024-01-16T15:01:28.565 INFO:teuthology.orchestra.run.smithi204.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-16T15:01:28.565 INFO:teuthology.orchestra.run.smithi204.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-16T15:01:28.565 INFO:teuthology.orchestra.run.smithi204.stdout: inet 127.0.0.1/8 scope host lo 2024-01-16T15:01:28.565 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:01:28.565 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 ::1/128 scope host 2024-01-16T15:01:28.566 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:01:28.566 INFO:teuthology.orchestra.run.smithi204.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-16T15:01:28.566 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2a brd ff:ff:ff:ff:ff:ff 2024-01-16T15:01:28.566 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f0 2024-01-16T15:01:28.566 INFO:teuthology.orchestra.run.smithi204.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-16T15:01:28.566 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2b brd ff:ff:ff:ff:ff:ff 2024-01-16T15:01:28.566 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f1 2024-01-16T15:01:28.566 INFO:teuthology.orchestra.run.smithi204.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-16T15:01:28.566 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:aa brd ff:ff:ff:ff:ff:ff 2024-01-16T15:01:28.566 INFO:teuthology.orchestra.run.smithi204.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-16T15:01:28.566 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:ab brd ff:ff:ff:ff:ff:ff 2024-01-16T15:01:28.566 INFO:teuthology.orchestra.run.smithi204.stdout: inet 172.21.15.204/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-16T15:01:28.567 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft 42555sec preferred_lft 42555sec 2024-01-16T15:01:28.567 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::ec4:7aff:fe88:77ab/64 scope link 2024-01-16T15:01:28.567 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:01:28.567 INFO:teuthology.orchestra.run.smithi204.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-16T15:01:28.567 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 02:22:3e:e0:99:fa brd ff:ff:ff:ff:ff:ff 2024-01-16T15:01:28.567 INFO:teuthology.orchestra.run.smithi204.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-16T15:01:28.567 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:01:28.567 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::30d5:f7ff:fe0d:9796/64 scope link 2024-01-16T15:01:28.567 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:01:28.567 INFO:teuthology.orchestra.run.smithi204.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-16T15:01:28.567 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 02:22:3e:e0:99:fa brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-16T15:01:28.567 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::22:3eff:fee0:99fa/64 scope link 2024-01-16T15:01:28.568 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:01:28.568 INFO:teuthology.orchestra.run.smithi204.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-16T15:01:28.568 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 76:f1:7f:74:a9:ca brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-16T15:01:28.568 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::74f1:7fff:fe74:a9ca/64 scope link 2024-01-16T15:01:28.568 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-16T15:01:28.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:01:28.569 DEBUG:teuthology.orchestra.run.smithi204:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-16T15:01:28.660 INFO:teuthology.orchestra.run.smithi204.stdout:1 2024-01-16T15:01:28.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:01:28.665 DEBUG:teuthology.orchestra.run.smithi204:> route 2024-01-16T15:01:28.717 INFO:teuthology.orchestra.run.smithi204.stdout:Kernel IP routing table 2024-01-16T15:01:28.717 INFO:teuthology.orchestra.run.smithi204.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-16T15:01:28.717 INFO:teuthology.orchestra.run.smithi204.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-16T15:01:28.717 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-16T15:01:28.717 INFO:teuthology.orchestra.run.smithi204.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-16T15:01:28.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:01:28.719 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec bash -c ' 2024-01-16T15:01:28.719 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-01-16T15:01:28.719 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-16T15:01:28.719 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-16T15:01:28.719 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-16T15:01:28.719 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-01-16T15:01:28.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:01:28.897 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-01-16T15:01:28.947 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-16T15:01:28.947 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-16T15:01:28.948 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-16T15:01:28.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:01:28.948 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set brx.0 up 2024-01-16T15:01:29.042 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-16T15:01:29.042 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-16T15:01:29.095 INFO:teuthology.orchestra.run.smithi204.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-16T15:01:29.095 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-16T15:01:29.095 DEBUG:teuthology.orchestra.run.smithi204:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-16T15:01:29.143 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-16T15:01:29.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:01:29.143 DEBUG:teuthology.orchestra.run.smithi204:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-16T15:01:29.277 INFO:teuthology.orchestra.run.smithi204.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-16T15:01:29.277 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-16T15:01:29.278 INFO:teuthology.orchestra.run.smithi204.stdout:invalid new device string format 2024-01-16T15:01:29.299 INFO:teuthology.orchestra.run.smithi204.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-16T15:01:29.299 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-16T15:01:29.299 INFO:teuthology.orchestra.run.smithi204.stdout:invalid new device string format 2024-01-16T15:01:29.299 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: resolved to: "172.21.15.143:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-01-16T15:01:29.299 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-16T15:01:29.328 INFO:teuthology.orchestra.run.smithi204.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-16T15:01:29.328 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-16T15:01:29.328 INFO:teuthology.orchestra.run.smithi204.stdout:invalid new device string format 2024-01-16T15:01:29.329 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: resolved to: "172.21.15.143:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-01-16T15:01:29.329 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-16T15:01:29.329 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: resolved to: "172.21.15.143:6789,172.21.15.204:6789,172.21.15.204:6790" 2024-01-16T15:01:29.329 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.143:6789,172.21.15.204:6789,172.21.15.204:6790:/ 2024-01-16T15:01:29.329 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0" will pass to kernel 2024-01-16T15:01:29.332 INFO:teuthology.orchestra.run.smithi204.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-01-16T15:01:29.332 INFO:teuthology.orchestra.run.smithi204.stdout: You just mounted a file system that supports labels which does not 2024-01-16T15:01:29.332 INFO:teuthology.orchestra.run.smithi204.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-01-16T15:01:29.332 INFO:teuthology.orchestra.run.smithi204.stdout: applications will generate AVC messages and not be allowed access to 2024-01-16T15:01:29.332 INFO:teuthology.orchestra.run.smithi204.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-01-16T15:01:29.338 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-16T15:01:29.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:01:29.338 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-16T15:01:29.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:01:29.414 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-16T15:01:29.463 INFO:teuthology.orchestra.run.smithi204.stdout:client5274 2024-01-16T15:01:29.463 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-16T15:01:29.464 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T15:01:29.508 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-01-16T15:01:29.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.744+0000 7fe130000640 1 Processor -- start 2024-01-16T15:01:29.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.744+0000 7fe130000640 1 -- start start 2024-01-16T15:01:29.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.746+0000 7fe130000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1280cc470 0x7fe1280abfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:29.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.746+0000 7fe130000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe1280cfb60 0x7fe1280b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:29.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.746+0000 7fe130000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1280ce270 con 0x7fe1280ce640 2024-01-16T15:01:29.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.746+0000 7fe130000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe12806c560 con 0x7fe1280cfb60 2024-01-16T15:01:29.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.746+0000 7fe130000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe12806d0a0 con 0x7fe1280cc470 2024-01-16T15:01:29.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.747+0000 7fe12e000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe1280cfb60 0x7fe1280b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:29.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.747+0000 7fe12e000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe1280cfb60 0x7fe1280b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:56610/0 (socket says 172.21.15.143:56610) 2024-01-16T15:01:29.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.747+0000 7fe12e000640 1 -- 172.21.15.143:0/4050819704 learned_addr learned my addr 172.21.15.143:0/4050819704 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:29.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.747+0000 7fe12cc00640 1 --1- 172.21.15.143:0/4050819704 >> v1:172.21.15.204:6789/0 conn(0x7fe1280ce640 0x7fe1280cf450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:48346/0 (socket says 172.21.15.143:48346) 2024-01-16T15:01:29.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.747+0000 7fe12d600640 1 --2- 172.21.15.143:0/4050819704 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1280cc470 0x7fe1280abfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:29.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.747+0000 7fe12e000640 1 -- 172.21.15.143:0/4050819704 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1280cc470 msgr2=0x7fe1280abfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:29.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.747+0000 7fe12e000640 1 --2- 172.21.15.143:0/4050819704 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1280cc470 0x7fe1280abfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:29.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.747+0000 7fe12e000640 1 -- 172.21.15.143:0/4050819704 >> v1:172.21.15.204:6789/0 conn(0x7fe1280ce640 legacy=0x7fe1280cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:29.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.747+0000 7fe12e000640 1 -- 172.21.15.143:0/4050819704 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1280b4200 con 0x7fe1280cfb60 2024-01-16T15:01:29.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.747+0000 7fe12d600640 1 --2- 172.21.15.143:0/4050819704 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1280cc470 0x7fe1280abfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T15:01:29.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.749+0000 7fe12e000640 1 --2- 172.21.15.143:0/4050819704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1280cfb60 0x7fe1280b3cc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe120009e80 tx=0x7fe12002bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a37ae10722b0e67 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:29.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.749+0000 7fe127e00640 1 -- 172.21.15.143:0/4050819704 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12002cae0 con 0x7fe1280cfb60 2024-01-16T15:01:29.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.749+0000 7fe127e00640 1 -- 172.21.15.143:0/4050819704 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe12002cc40 con 0x7fe1280cfb60 2024-01-16T15:01:29.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.749+0000 7fe127e00640 1 -- 172.21.15.143:0/4050819704 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1200343f0 con 0x7fe1280cfb60 2024-01-16T15:01:29.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.750+0000 7fe130000640 1 -- 172.21.15.143:0/4050819704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1280cfb60 msgr2=0x7fe1280b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:29.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.750+0000 7fe130000640 1 --2- 172.21.15.143:0/4050819704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1280cfb60 0x7fe1280b3cc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe120009e80 tx=0x7fe12002bf40 comp rx=0 tx=0).stop 2024-01-16T15:01:29.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.750+0000 7fe130000640 1 -- 172.21.15.143:0/4050819704 shutdown_connections 2024-01-16T15:01:29.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.750+0000 7fe130000640 1 --2- 172.21.15.143:0/4050819704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1280cfb60 0x7fe1280b3cc0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:29.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.750+0000 7fe130000640 1 --2- 172.21.15.143:0/4050819704 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1280cc470 0x7fe1280abfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:29.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.750+0000 7fe130000640 1 -- 172.21.15.143:0/4050819704 >> 172.21.15.143:0/4050819704 conn(0x7fe12806a920 msgr2=0x7fe12806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:29.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.750+0000 7fe130000640 1 -- 172.21.15.143:0/4050819704 shutdown_connections 2024-01-16T15:01:29.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.750+0000 7fe130000640 1 -- 172.21.15.143:0/4050819704 wait complete. 2024-01-16T15:01:29.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.752+0000 7fe130000640 1 Processor -- start 2024-01-16T15:01:29.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.752+0000 7fe130000640 1 -- start start 2024-01-16T15:01:29.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.753+0000 7fe130000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1280cc470 0x7fe12814f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:29.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.753+0000 7fe130000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1280ce640 0x7fe12814be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:29.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.753+0000 7fe130000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1280cfb60 0x7fe12815e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:29.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.753+0000 7fe130000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12814c830 con 0x7fe1280ce640 2024-01-16T15:01:29.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.753+0000 7fe130000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1281624b0 con 0x7fe1280cfb60 2024-01-16T15:01:29.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.753+0000 7fe130000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe12814cad0 con 0x7fe1280cc470 2024-01-16T15:01:29.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.753+0000 7fe12cc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1280ce640 0x7fe12814be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:29.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.753+0000 7fe12cc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1280ce640 0x7fe12814be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41028/0 (socket says 172.21.15.143:41028) 2024-01-16T15:01:29.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.753+0000 7fe12cc00640 1 -- 172.21.15.143:0/1346877432 learned_addr learned my addr 172.21.15.143:0/1346877432 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:29.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.754+0000 7fe12e000640 1 --2- 172.21.15.143:0/1346877432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1280cfb60 0x7fe12815e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:29.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.754+0000 7fe12d600640 1 --2- 172.21.15.143:0/1346877432 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1280cc470 0x7fe12814f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:29.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.754+0000 7fe12cc00640 1 -- 172.21.15.143:0/1346877432 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1280cc470 msgr2=0x7fe12814f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:29.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.754+0000 7fe12cc00640 1 --2- 172.21.15.143:0/1346877432 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1280cc470 0x7fe12814f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:29.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.754+0000 7fe12cc00640 1 -- 172.21.15.143:0/1346877432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1280cfb60 msgr2=0x7fe12815e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:29.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.754+0000 7fe12cc00640 1 --2- 172.21.15.143:0/1346877432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1280cfb60 0x7fe12815e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:29.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.754+0000 7fe12cc00640 1 -- 172.21.15.143:0/1346877432 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe118009400 con 0x7fe1280ce640 2024-01-16T15:01:29.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.754+0000 7fe12cc00640 1 --2- 172.21.15.143:0/1346877432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1280ce640 0x7fe12814be20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe11000c4d0 tx=0x7fe11000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:29.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.755+0000 7fe125600640 1 -- 172.21.15.143:0/1346877432 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe110007480 con 0x7fe1280ce640 2024-01-16T15:01:29.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.755+0000 7fe125600640 1 -- 172.21.15.143:0/1346877432 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1100075e0 con 0x7fe1280ce640 2024-01-16T15:01:29.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.755+0000 7fe125600640 1 -- 172.21.15.143:0/1346877432 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1100049e0 con 0x7fe1280ce640 2024-01-16T15:01:29.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.755+0000 7fe130000640 1 -- 172.21.15.143:0/1346877432 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe120009b30 con 0x7fe1280ce640 2024-01-16T15:01:29.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.755+0000 7fe130000640 1 -- 172.21.15.143:0/1346877432 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe12814f700 con 0x7fe1280ce640 2024-01-16T15:01:29.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.756+0000 7fe130000640 1 -- 172.21.15.143:0/1346877432 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe128067470 con 0x7fe1280ce640 2024-01-16T15:01:29.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.757+0000 7fe125600640 1 -- 172.21.15.143:0/1346877432 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fe110004b40 con 0x7fe1280ce640 2024-01-16T15:01:29.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.758+0000 7fe125600640 1 --2- 172.21.15.143:0/1346877432 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe10006b2c0 0x7fe10006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:29.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.758+0000 7fe125600640 1 -- 172.21.15.143:0/1346877432 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe110088cf0 con 0x7fe1280ce640 2024-01-16T15:01:29.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.758+0000 7fe12d600640 1 --2- 172.21.15.143:0/1346877432 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe10006b2c0 0x7fe10006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:29.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.759+0000 7fe12d600640 1 --2- 172.21.15.143:0/1346877432 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe10006b2c0 0x7fe10006d780 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe118009c60 tx=0x7fe118008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:29.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.763+0000 7fe125600640 1 -- 172.21.15.143:0/1346877432 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe110053b70 con 0x7fe1280ce640 2024-01-16T15:01:29.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:29.995+0000 7fe130000640 1 -- 172.21.15.143:0/1346877432 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe1280cd910 con 0x7fe1280ce640 2024-01-16T15:01:30.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.736+0000 7fe125600640 1 -- 172.21.15.143:0/1346877432 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 136+0+0 (secure 0 0 0) 0x7fe1100180c0 con 0x7fe1280ce640 2024-01-16T15:01:30.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.740+0000 7fe130000640 1 -- 172.21.15.143:0/1346877432 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe10006b2c0 msgr2=0x7fe10006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:30.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.740+0000 7fe130000640 1 --2- 172.21.15.143:0/1346877432 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe10006b2c0 0x7fe10006d780 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe118009c60 tx=0x7fe118008040 comp rx=0 tx=0).stop 2024-01-16T15:01:30.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.740+0000 7fe130000640 1 -- 172.21.15.143:0/1346877432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1280ce640 msgr2=0x7fe12814be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:30.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.740+0000 7fe130000640 1 --2- 172.21.15.143:0/1346877432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1280ce640 0x7fe12814be20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe11000c4d0 tx=0x7fe11000c990 comp rx=0 tx=0).stop 2024-01-16T15:01:30.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.740+0000 7fe130000640 1 -- 172.21.15.143:0/1346877432 shutdown_connections 2024-01-16T15:01:30.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.740+0000 7fe130000640 1 --2- 172.21.15.143:0/1346877432 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fe10006b2c0 0x7fe10006d780 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:30.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.740+0000 7fe130000640 1 --2- 172.21.15.143:0/1346877432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1280cfb60 0x7fe12815e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:30.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.740+0000 7fe130000640 1 --2- 172.21.15.143:0/1346877432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1280ce640 0x7fe12814be20 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:30.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.740+0000 7fe130000640 1 --2- 172.21.15.143:0/1346877432 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1280cc470 0x7fe12814f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:30.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.740+0000 7fe130000640 1 -- 172.21.15.143:0/1346877432 >> 172.21.15.143:0/1346877432 conn(0x7fe12806a920 msgr2=0x7fe1280ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:30.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.740+0000 7fe130000640 1 -- 172.21.15.143:0/1346877432 shutdown_connections 2024-01-16T15:01:30.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.740+0000 7fe130000640 1 -- 172.21.15.143:0/1346877432 wait complete. 2024-01-16T15:01:30.743 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-16T15:01:30.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.974+0000 7f2472c00640 1 Processor -- start 2024-01-16T15:01:30.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.975+0000 7f2472c00640 1 -- start start 2024-01-16T15:01:30.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.976+0000 7f2472c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f246c15d490 0x7f246c0cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:30.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.976+0000 7f2472c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f246c0cea30 con 0x7f246c0b63d0 2024-01-16T15:01:30.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.976+0000 7f2472c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f246c0ce750 con 0x7f246c0d3940 2024-01-16T15:01:30.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.976+0000 7f2472c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f246c0d0450 con 0x7f246c15d490 2024-01-16T15:01:30.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.976+0000 7f2470e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f246c0b63d0 0x7f246c0d3230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:58512/0 (socket says 172.21.15.143:58512) 2024-01-16T15:01:30.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.976+0000 7f2470e00640 1 -- 172.21.15.143:0/1670733805 learned_addr learned my addr 172.21.15.143:0/1670733805 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:01:30.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.977+0000 7f2471800640 1 --2- 172.21.15.143:0/1670733805 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f246c15d490 0x7f246c0cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:30.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.977+0000 7f246be00640 1 -- 172.21.15.143:0/1670733805 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3779177004 0 0) 0x7f246c0cea30 con 0x7f246c0b63d0 2024-01-16T15:01:30.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.977+0000 7f246be00640 1 -- 172.21.15.143:0/1670733805 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f244c003520 con 0x7f246c0b63d0 2024-01-16T15:01:30.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.977+0000 7f246be00640 1 -- 172.21.15.143:0/1670733805 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3857473088 0 0) 0x7f244c003520 con 0x7f246c0b63d0 2024-01-16T15:01:30.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.977+0000 7f246be00640 1 -- 172.21.15.143:0/1670733805 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f246c15d490 msgr2=0x7f246c0cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:30.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.977+0000 7f246be00640 1 --2- 172.21.15.143:0/1670733805 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f246c15d490 0x7f246c0cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:30.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.977+0000 7f246be00640 1 -- 172.21.15.143:0/1670733805 >> v1:172.21.15.204:6789/0 conn(0x7f246c0d3940 legacy=0x7f246c0d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:30.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.977+0000 7f246be00640 1 -- 172.21.15.143:0/1670733805 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f246c0d0630 con 0x7f246c0b63d0 2024-01-16T15:01:30.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.977+0000 7f246be00640 1 -- 172.21.15.143:0/1670733805 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7f2454001670 con 0x7f246c0b63d0 2024-01-16T15:01:30.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.977+0000 7f246be00640 1 -- 172.21.15.143:0/1670733805 >> v1:172.21.15.143:6789/0 conn(0x7f246c0b63d0 legacy=0x7f246c0d3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:30.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.977+0000 7f246be00640 1 --2- 172.21.15.143:0/1670733805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f244c0039a0 0x7f244c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:30.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.977+0000 7f246be00640 1 --2- 172.21.15.143:0/1670733805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f244c0047c0 0x7f244c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:30.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.977+0000 7f246be00640 1 --2- 172.21.15.143:0/1670733805 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f246c15d490 0x7f244c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:30.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.977+0000 7f246be00640 1 -- 172.21.15.143:0/1670733805 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f246c0d0450 con 0x7f244c0039a0 2024-01-16T15:01:30.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.977+0000 7f246be00640 1 -- 172.21.15.143:0/1670733805 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244c00ba60 con 0x7f244c0047c0 2024-01-16T15:01:30.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.977+0000 7f246be00640 1 -- 172.21.15.143:0/1670733805 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f244c00bc20 con 0x7f246c15d490 2024-01-16T15:01:30.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.978+0000 7f2471800640 1 --2- 172.21.15.143:0/1670733805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f244c0039a0 0x7f244c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:30.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.978+0000 7f2470e00640 1 --2- 172.21.15.143:0/1670733805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f244c0047c0 0x7f244c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:30.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.978+0000 7f2471800640 1 -- 172.21.15.143:0/1670733805 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f246c15d490 msgr2=0x7f244c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:30.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.978+0000 7f2471800640 1 --2- 172.21.15.143:0/1670733805 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f246c15d490 0x7f244c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:30.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.978+0000 7f2471800640 1 -- 172.21.15.143:0/1670733805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f244c0047c0 msgr2=0x7f244c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:30.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.978+0000 7f2471800640 1 --2- 172.21.15.143:0/1670733805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f244c0047c0 0x7f244c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:30.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.978+0000 7f2471800640 1 -- 172.21.15.143:0/1670733805 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f244c003520 con 0x7f244c0039a0 2024-01-16T15:01:30.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.979+0000 7f2471800640 1 --2- 172.21.15.143:0/1670733805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f244c0039a0 0x7f244c003dc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f245c009a90 tx=0x7f244c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebce221ea511499 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:30.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.979+0000 7f246be00640 1 -- 172.21.15.143:0/1670733805 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245c02c690 con 0x7f244c0039a0 2024-01-16T15:01:30.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.979+0000 7f246be00640 1 -- 172.21.15.143:0/1670733805 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f245c02c7f0 con 0x7f244c0039a0 2024-01-16T15:01:30.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.979+0000 7f246be00640 1 -- 172.21.15.143:0/1670733805 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245c02cb80 con 0x7f244c0039a0 2024-01-16T15:01:30.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.980+0000 7f2472c00640 1 -- 172.21.15.143:0/1670733805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f244c0039a0 msgr2=0x7f244c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:30.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.980+0000 7f2472c00640 1 --2- 172.21.15.143:0/1670733805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f244c0039a0 0x7f244c003dc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f245c009a90 tx=0x7f244c0057f0 comp rx=0 tx=0).stop 2024-01-16T15:01:30.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.980+0000 7f2472200640 1 -- 172.21.15.143:0/1670733805 reap_dead start 2024-01-16T15:01:30.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.980+0000 7f2472c00640 1 -- 172.21.15.143:0/1670733805 shutdown_connections 2024-01-16T15:01:30.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.980+0000 7f2472c00640 1 -- 172.21.15.143:0/1670733805 >> 172.21.15.143:0/1670733805 conn(0x7f246c061340 msgr2=0x7f246c061750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:30.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.980+0000 7f2472c00640 1 -- 172.21.15.143:0/1670733805 shutdown_connections 2024-01-16T15:01:30.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.980+0000 7f2472c00640 1 -- 172.21.15.143:0/1670733805 wait complete. 2024-01-16T15:01:30.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.982+0000 7f2472c00640 1 Processor -- start 2024-01-16T15:01:30.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.982+0000 7f2472c00640 1 -- start start 2024-01-16T15:01:30.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.983+0000 7f2472c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f244c0039a0 0x7f246c1e7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:30.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.983+0000 7f2472c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f246c154b70 0x7f246c1e9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:30.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.983+0000 7f2472c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f246c1ea510 0x7f246c0c8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:30.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.983+0000 7f2472c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f246c0d0bf0 con 0x7f246c1ea510 2024-01-16T15:01:30.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.983+0000 7f2472c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f246c15e270 con 0x7f246c154b70 2024-01-16T15:01:30.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.983+0000 7f2472c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f246c1ea990 con 0x7f244c0039a0 2024-01-16T15:01:30.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.983+0000 7f2472200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f246c1ea510 0x7f246c0c8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:30.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.983+0000 7f2472200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f246c1ea510 0x7f246c0c8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41058/0 (socket says 172.21.15.143:41058) 2024-01-16T15:01:30.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.983+0000 7f2472200640 1 -- 172.21.15.143:0/1492670759 learned_addr learned my addr 172.21.15.143:0/1492670759 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:30.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.983+0000 7f2471800640 1 --2- 172.21.15.143:0/1492670759 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f244c0039a0 0x7f246c1e7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:30.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.983+0000 7f2470e00640 1 --2- 172.21.15.143:0/1492670759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f246c154b70 0x7f246c1e9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:30.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.984+0000 7f2472200640 1 -- 172.21.15.143:0/1492670759 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f244c0039a0 msgr2=0x7f246c1e7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:30.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.984+0000 7f2472200640 1 --2- 172.21.15.143:0/1492670759 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f244c0039a0 0x7f246c1e7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:30.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.984+0000 7f2472200640 1 -- 172.21.15.143:0/1492670759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f246c154b70 msgr2=0x7f246c1e9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:30.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.984+0000 7f2472200640 1 --2- 172.21.15.143:0/1492670759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f246c154b70 0x7f246c1e9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:30.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.984+0000 7f2472200640 1 -- 172.21.15.143:0/1492670759 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f246c158b70 con 0x7f246c1ea510 2024-01-16T15:01:30.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.984+0000 7f2472200640 1 --2- 172.21.15.143:0/1492670759 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f246c1ea510 0x7f246c0c8b90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f246c161f50 tx=0x7f246400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:30.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.984+0000 7f2469600640 1 -- 172.21.15.143:0/1492670759 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2464005d40 con 0x7f246c1ea510 2024-01-16T15:01:30.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.984+0000 7f2469600640 1 -- 172.21.15.143:0/1492670759 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f246400ced0 con 0x7f246c1ea510 2024-01-16T15:01:30.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.984+0000 7f2469600640 1 -- 172.21.15.143:0/1492670759 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246400a880 con 0x7f246c1ea510 2024-01-16T15:01:30.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.984+0000 7f2472c00640 1 -- 172.21.15.143:0/1492670759 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f246c15d910 con 0x7f246c1ea510 2024-01-16T15:01:30.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.984+0000 7f2472c00640 1 -- 172.21.15.143:0/1492670759 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f246c15de90 con 0x7f246c1ea510 2024-01-16T15:01:30.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.986+0000 7f2472c00640 1 -- 172.21.15.143:0/1492670759 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f246c15e3b0 con 0x7f246c1ea510 2024-01-16T15:01:30.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.986+0000 7f2469600640 1 -- 172.21.15.143:0/1492670759 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f246400a9e0 con 0x7f246c1ea510 2024-01-16T15:01:30.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.987+0000 7f2469600640 1 --2- 172.21.15.143:0/1492670759 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f244406b2f0 0x7f244406d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:30.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.987+0000 7f2469600640 1 -- 172.21.15.143:0/1492670759 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2464088a30 con 0x7f246c1ea510 2024-01-16T15:01:30.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.988+0000 7f2471800640 1 --2- 172.21.15.143:0/1492670759 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f244406b2f0 0x7f244406d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:30.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.988+0000 7f2471800640 1 --2- 172.21.15.143:0/1492670759 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f244406b2f0 0x7f244406d7b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f245c036000 tx=0x7f245c030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:30.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:30.992+0000 7f2469600640 1 -- 172.21.15.143:0/1492670759 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f246c15e3b0 con 0x7f246c1ea510 2024-01-16T15:01:31.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.204+0000 7f2472c00640 1 -- 172.21.15.143:0/1492670759 --> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f246c0cd8f0 con 0x7f244406b2f0 2024-01-16T15:01:31.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.207+0000 7f2469600640 1 -- 172.21.15.143:0/1492670759 <== mgr.4113 v2:172.21.15.143:6832/3153492115 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f245c004040 con 0x7f244406b2f0 2024-01-16T15:01:31.188 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-16T15:01:31.188 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-16T15:01:31.188 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cephfs" 2024-01-16T15:01:31.188 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:01:31.188 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-16T15:01:31.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.210+0000 7f2472c00640 1 -- 172.21.15.143:0/1492670759 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f244406b2f0 msgr2=0x7f244406d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:31.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.210+0000 7f2472c00640 1 --2- 172.21.15.143:0/1492670759 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f244406b2f0 0x7f244406d7b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f245c036000 tx=0x7f245c030000 comp rx=0 tx=0).stop 2024-01-16T15:01:31.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.210+0000 7f2472c00640 1 -- 172.21.15.143:0/1492670759 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f246c1ea510 msgr2=0x7f246c0c8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:31.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.210+0000 7f2472c00640 1 --2- 172.21.15.143:0/1492670759 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f246c1ea510 0x7f246c0c8b90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f246c161f50 tx=0x7f246400cb90 comp rx=0 tx=0).stop 2024-01-16T15:01:31.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.210+0000 7f2472c00640 1 -- 172.21.15.143:0/1492670759 shutdown_connections 2024-01-16T15:01:31.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.210+0000 7f2472c00640 1 --2- 172.21.15.143:0/1492670759 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f244406b2f0 0x7f244406d7b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:31.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.210+0000 7f2472c00640 1 --2- 172.21.15.143:0/1492670759 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f246c1ea510 0x7f246c0c8b90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:31.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.210+0000 7f2472c00640 1 --2- 172.21.15.143:0/1492670759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f246c154b70 0x7f246c1e9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:31.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.210+0000 7f2472c00640 1 --2- 172.21.15.143:0/1492670759 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f244c0039a0 0x7f246c1e7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:31.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.210+0000 7f2472c00640 1 -- 172.21.15.143:0/1492670759 >> 172.21.15.143:0/1492670759 conn(0x7f246c061340 msgr2=0x7f246c160cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:31.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.210+0000 7f2472c00640 1 -- 172.21.15.143:0/1492670759 shutdown_connections 2024-01-16T15:01:31.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.210+0000 7f2472c00640 1 -- 172.21.15.143:0/1492670759 wait complete. 2024-01-16T15:01:31.208 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-16T15:01:31.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.440+0000 7fdce9600640 1 Processor -- start 2024-01-16T15:01:31.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.440+0000 7fdce9600640 1 -- start start 2024-01-16T15:01:31.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.442+0000 7fdce9600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdce4072160 0x7fdce415e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:31.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.442+0000 7fdce9600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdce406de10 0x7fdce40705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:31.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.442+0000 7fdce9600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdce415ec90 con 0x7fdce40c49f0 2024-01-16T15:01:31.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.442+0000 7fdce9600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdce40cd530 con 0x7fdce406de10 2024-01-16T15:01:31.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.442+0000 7fdce9600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdce40c4ed0 con 0x7fdce4072160 2024-01-16T15:01:31.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.442+0000 7fdce3400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdce406de10 0x7fdce40705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:31.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.442+0000 7fdce3400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdce406de10 0x7fdce40705c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41060/0 (socket says 172.21.15.143:41060) 2024-01-16T15:01:31.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.442+0000 7fdce3400640 1 -- 172.21.15.143:0/1736835040 learned_addr learned my addr 172.21.15.143:0/1736835040 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:31.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.442+0000 7fdce2a00640 1 --2- 172.21.15.143:0/1736835040 >> v2:172.21.15.204:3300/0 conn(0x7fdce4072160 0x7fdce415e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:31.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.443+0000 7fdce3400640 1 -- 172.21.15.143:0/1736835040 >> v2:172.21.15.204:3300/0 conn(0x7fdce4072160 msgr2=0x7fdce415e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:31.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.443+0000 7fdce3400640 1 --2- 172.21.15.143:0/1736835040 >> v2:172.21.15.204:3300/0 conn(0x7fdce4072160 0x7fdce415e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:31.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.443+0000 7fdce3400640 1 -- 172.21.15.143:0/1736835040 >> v1:172.21.15.143:6789/0 conn(0x7fdce40c49f0 legacy=0x7fdce406d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:31.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.443+0000 7fdce3400640 1 -- 172.21.15.143:0/1736835040 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce40c51b0 con 0x7fdce406de10 2024-01-16T15:01:31.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.444+0000 7fdce3400640 1 --2- 172.21.15.143:0/1736835040 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdce406de10 0x7fdce40705c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdce40ccae0 tx=0x7fdcdc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a926c853bfdf61 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:31.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.444+0000 7fdce2000640 1 -- 172.21.15.143:0/1736835040 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc030860 con 0x7fdce406de10 2024-01-16T15:01:31.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.444+0000 7fdce2000640 1 -- 172.21.15.143:0/1736835040 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcdc0309c0 con 0x7fdce406de10 2024-01-16T15:01:31.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.444+0000 7fdce2000640 1 -- 172.21.15.143:0/1736835040 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc030d50 con 0x7fdce406de10 2024-01-16T15:01:31.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.445+0000 7fdce9600640 1 -- 172.21.15.143:0/1736835040 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdce406de10 msgr2=0x7fdce40705c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:31.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.445+0000 7fdce9600640 1 --2- 172.21.15.143:0/1736835040 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdce406de10 0x7fdce40705c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdce40ccae0 tx=0x7fdcdc02b9a0 comp rx=0 tx=0).stop 2024-01-16T15:01:31.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.445+0000 7fdce9600640 1 -- 172.21.15.143:0/1736835040 shutdown_connections 2024-01-16T15:01:31.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.445+0000 7fdce9600640 1 --2- 172.21.15.143:0/1736835040 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdce406de10 0x7fdce40705c0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:31.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.445+0000 7fdce9600640 1 --2- 172.21.15.143:0/1736835040 >> v2:172.21.15.204:3300/0 conn(0x7fdce4072160 0x7fdce415e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:31.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.445+0000 7fdce9600640 1 -- 172.21.15.143:0/1736835040 >> 172.21.15.143:0/1736835040 conn(0x7fdce40b0c50 msgr2=0x7fdce40b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:31.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.445+0000 7fdce9600640 1 -- 172.21.15.143:0/1736835040 shutdown_connections 2024-01-16T15:01:31.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.445+0000 7fdce9600640 1 -- 172.21.15.143:0/1736835040 wait complete. 2024-01-16T15:01:31.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.446+0000 7fdce9600640 1 Processor -- start 2024-01-16T15:01:31.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.447+0000 7fdce9600640 1 -- start start 2024-01-16T15:01:31.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.448+0000 7fdce9600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdce406de10 0x7fdce40cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:31.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.448+0000 7fdce9600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdce4072160 0x7fdce414eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:31.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.448+0000 7fdce9600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdce40c49f0 0x7fdce409b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:31.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.448+0000 7fdce9600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce40cba20 con 0x7fdce40c49f0 2024-01-16T15:01:31.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.448+0000 7fdce9600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce40cae10 con 0x7fdce4072160 2024-01-16T15:01:31.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.448+0000 7fdce9600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce40cb0f0 con 0x7fdce406de10 2024-01-16T15:01:31.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.448+0000 7fdce3400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdce40c49f0 0x7fdce409b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:31.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.448+0000 7fdce3400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdce40c49f0 0x7fdce409b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41066/0 (socket says 172.21.15.143:41066) 2024-01-16T15:01:31.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.448+0000 7fdce3400640 1 -- 172.21.15.143:0/3194624125 learned_addr learned my addr 172.21.15.143:0/3194624125 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:31.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.448+0000 7fdce2a00640 1 --2- 172.21.15.143:0/3194624125 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdce406de10 0x7fdce40cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:31.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.448+0000 7fdcd9e00640 1 --2- 172.21.15.143:0/3194624125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdce4072160 0x7fdce414eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:31.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.448+0000 7fdce3400640 1 -- 172.21.15.143:0/3194624125 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdce406de10 msgr2=0x7fdce40cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:31.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.448+0000 7fdce3400640 1 --2- 172.21.15.143:0/3194624125 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdce406de10 0x7fdce40cbeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:31.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.448+0000 7fdce3400640 1 -- 172.21.15.143:0/3194624125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdce4072160 msgr2=0x7fdce414eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:31.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.448+0000 7fdce3400640 1 --2- 172.21.15.143:0/3194624125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdce4072160 0x7fdce414eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:31.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.448+0000 7fdce3400640 1 -- 172.21.15.143:0/3194624125 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcdc0094d0 con 0x7fdce40c49f0 2024-01-16T15:01:31.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.448+0000 7fdcd9e00640 1 --2- 172.21.15.143:0/3194624125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdce4072160 0x7fdce414eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:01:31.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.449+0000 7fdce3400640 1 --2- 172.21.15.143:0/3194624125 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdce40c49f0 0x7fdce409b0b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0090b0 tx=0x7fdcdc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:31.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.449+0000 7fdcdb400640 1 -- 172.21.15.143:0/3194624125 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc039c90 con 0x7fdce40c49f0 2024-01-16T15:01:31.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.449+0000 7fdcdb400640 1 -- 172.21.15.143:0/3194624125 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcdc039df0 con 0x7fdce40c49f0 2024-01-16T15:01:31.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.449+0000 7fdcdb400640 1 -- 172.21.15.143:0/3194624125 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc043640 con 0x7fdce40c49f0 2024-01-16T15:01:31.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.449+0000 7fdce9600640 1 -- 172.21.15.143:0/3194624125 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdce40cb3d0 con 0x7fdce40c49f0 2024-01-16T15:01:31.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.449+0000 7fdce9600640 1 -- 172.21.15.143:0/3194624125 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdce414f060 con 0x7fdce40c49f0 2024-01-16T15:01:31.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.450+0000 7fdce9600640 1 -- 172.21.15.143:0/3194624125 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdce40cbbd0 con 0x7fdce40c49f0 2024-01-16T15:01:31.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.451+0000 7fdcdb400640 1 -- 172.21.15.143:0/3194624125 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fdcdc0437a0 con 0x7fdce40c49f0 2024-01-16T15:01:31.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.452+0000 7fdcdb400640 1 --2- 172.21.15.143:0/3194624125 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fdcbc06b2f0 0x7fdcbc06d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:31.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.452+0000 7fdcdb400640 1 -- 172.21.15.143:0/3194624125 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdcdc03e070 con 0x7fdce40c49f0 2024-01-16T15:01:31.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.452+0000 7fdce2a00640 1 --2- 172.21.15.143:0/3194624125 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fdcbc06b2f0 0x7fdcbc06d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:31.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.453+0000 7fdce2a00640 1 --2- 172.21.15.143:0/3194624125 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fdcbc06b2f0 0x7fdcbc06d7b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4006fd0 tx=0x7fdcd4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:31.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.456+0000 7fdcdb400640 1 -- 172.21.15.143:0/3194624125 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdce40cbbd0 con 0x7fdce40c49f0 2024-01-16T15:01:31.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.647+0000 7fdce9600640 1 -- 172.21.15.143:0/3194624125 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fdce40705c0 con 0x7fdce40c49f0 2024-01-16T15:01:31.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.647+0000 7fdcdb400640 1 -- 172.21.15.143:0/3194624125 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdcdc0340c0 con 0x7fdce40c49f0 2024-01-16T15:01:31.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.650+0000 7fdce9600640 1 -- 172.21.15.143:0/3194624125 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fdcbc06b2f0 msgr2=0x7fdcbc06d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:31.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.650+0000 7fdce9600640 1 --2- 172.21.15.143:0/3194624125 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fdcbc06b2f0 0x7fdcbc06d7b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4006fd0 tx=0x7fdcd4008040 comp rx=0 tx=0).stop 2024-01-16T15:01:31.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.650+0000 7fdce9600640 1 -- 172.21.15.143:0/3194624125 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdce40c49f0 msgr2=0x7fdce409b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:31.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.650+0000 7fdce9600640 1 --2- 172.21.15.143:0/3194624125 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdce40c49f0 0x7fdce409b0b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0090b0 tx=0x7fdcdc004730 comp rx=0 tx=0).stop 2024-01-16T15:01:31.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.650+0000 7fdce9600640 1 -- 172.21.15.143:0/3194624125 shutdown_connections 2024-01-16T15:01:31.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.650+0000 7fdce9600640 1 --2- 172.21.15.143:0/3194624125 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fdcbc06b2f0 0x7fdcbc06d7b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:31.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.650+0000 7fdce9600640 1 --2- 172.21.15.143:0/3194624125 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdce40c49f0 0x7fdce409b0b0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:31.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.650+0000 7fdce9600640 1 --2- 172.21.15.143:0/3194624125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdce4072160 0x7fdce414eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:31.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.650+0000 7fdce9600640 1 --2- 172.21.15.143:0/3194624125 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdce406de10 0x7fdce40cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:31.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.650+0000 7fdce9600640 1 -- 172.21.15.143:0/3194624125 >> 172.21.15.143:0/3194624125 conn(0x7fdce40b0c50 msgr2=0x7fdce40ac4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:31.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.651+0000 7fdce9600640 1 -- 172.21.15.143:0/3194624125 shutdown_connections 2024-01-16T15:01:31.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.651+0000 7fdce9600640 1 -- 172.21.15.143:0/3194624125 wait complete. 2024-01-16T15:01:31.649 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-01-16T15:01:31.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.870+0000 7fd93f200640 1 Processor -- start 2024-01-16T15:01:31.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.870+0000 7fd93f200640 1 -- start start 2024-01-16T15:01:31.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.872+0000 7fd93f200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd93815d260 0x7fd9380cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:31.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.872+0000 7fd93f200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9380d17b0 0x7fd9380d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:31.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.872+0000 7fd93f200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9380ce280 con 0x7fd9380cfc30 2024-01-16T15:01:31.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.872+0000 7fd93f200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9380ce4d0 con 0x7fd93815d260 2024-01-16T15:01:31.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.872+0000 7fd93f200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9380ce7b0 con 0x7fd9380d17b0 2024-01-16T15:01:31.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.872+0000 7fd93e800640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd9380cfc30 0x7fd9380d0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:58530/0 (socket says 172.21.15.143:58530) 2024-01-16T15:01:31.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.872+0000 7fd93e800640 1 -- 172.21.15.143:0/2014460704 learned_addr learned my addr 172.21.15.143:0/2014460704 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:01:31.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.873+0000 7fd93de00640 1 --2- 172.21.15.143:0/2014460704 >> v2:172.21.15.204:3300/0 conn(0x7fd93815d260 0x7fd9380cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:31.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.873+0000 7fd93d400640 1 --2- 172.21.15.143:0/2014460704 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9380d17b0 0x7fd9380d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:31.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.873+0000 7fd93ca00640 1 -- 172.21.15.143:0/2014460704 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 779389803 0 0) 0x7fd9380ce280 con 0x7fd9380cfc30 2024-01-16T15:01:31.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.873+0000 7fd93ca00640 1 -- 172.21.15.143:0/2014460704 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd918003520 con 0x7fd9380cfc30 2024-01-16T15:01:31.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.873+0000 7fd93ca00640 1 -- 172.21.15.143:0/2014460704 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1308190150 0 0) 0x7fd918003520 con 0x7fd9380cfc30 2024-01-16T15:01:31.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.873+0000 7fd93ca00640 1 -- 172.21.15.143:0/2014460704 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9380d17b0 msgr2=0x7fd9380d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:31.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.873+0000 7fd93ca00640 1 --2- 172.21.15.143:0/2014460704 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9380d17b0 0x7fd9380d1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:31.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.873+0000 7fd93ca00640 1 -- 172.21.15.143:0/2014460704 >> v2:172.21.15.204:3300/0 conn(0x7fd93815d260 msgr2=0x7fd9380cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:31.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.873+0000 7fd93ca00640 1 --2- 172.21.15.143:0/2014460704 >> v2:172.21.15.204:3300/0 conn(0x7fd93815d260 0x7fd9380cdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:31.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.873+0000 7fd93ca00640 1 -- 172.21.15.143:0/2014460704 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9380b8b40 con 0x7fd9380cfc30 2024-01-16T15:01:31.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.873+0000 7fd93ca00640 1 -- 172.21.15.143:0/2014460704 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7fd9300019f0 con 0x7fd9380cfc30 2024-01-16T15:01:31.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.873+0000 7fd93ca00640 1 -- 172.21.15.143:0/2014460704 >> v1:172.21.15.143:6789/0 conn(0x7fd9380cfc30 legacy=0x7fd9380d0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:31.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.873+0000 7fd93ca00640 1 --2- 172.21.15.143:0/2014460704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd918003970 0x7fd918003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:31.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.873+0000 7fd93ca00640 1 --2- 172.21.15.143:0/2014460704 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9380d17b0 0x7fd9180047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:31.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.873+0000 7fd93ca00640 1 --2- 172.21.15.143:0/2014460704 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd918004d00 0x7fd918005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:31.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.873+0000 7fd93ca00640 1 -- 172.21.15.143:0/2014460704 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9380ce4d0 con 0x7fd918004d00 2024-01-16T15:01:31.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.873+0000 7fd93ca00640 1 -- 172.21.15.143:0/2014460704 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9380ce7b0 con 0x7fd918003970 2024-01-16T15:01:31.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.873+0000 7fd93ca00640 1 -- 172.21.15.143:0/2014460704 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd91800b980 con 0x7fd9380d17b0 2024-01-16T15:01:31.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.874+0000 7fd93e800640 1 --2- 172.21.15.143:0/2014460704 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd918004d00 0x7fd918005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:31.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.874+0000 7fd93de00640 1 --2- 172.21.15.143:0/2014460704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd918003970 0x7fd918003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:31.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.874+0000 7fd93d400640 1 --2- 172.21.15.143:0/2014460704 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9380d17b0 0x7fd9180047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:31.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.874+0000 7fd93e800640 1 -- 172.21.15.143:0/2014460704 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9380d17b0 msgr2=0x7fd9180047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:31.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.874+0000 7fd93e800640 1 --2- 172.21.15.143:0/2014460704 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9380d17b0 0x7fd9180047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:31.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.874+0000 7fd93e800640 1 -- 172.21.15.143:0/2014460704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd918003970 msgr2=0x7fd918003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:31.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.874+0000 7fd93e800640 1 --2- 172.21.15.143:0/2014460704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd918003970 0x7fd918003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:31.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.874+0000 7fd93e800640 1 -- 172.21.15.143:0/2014460704 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd918003520 con 0x7fd918004d00 2024-01-16T15:01:31.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.875+0000 7fd93e800640 1 --2- 172.21.15.143:0/2014460704 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd918004d00 0x7fd918005180 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd93000a890 tx=0x7fd9180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75aa5effb1de36f1 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:31.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.875+0000 7fd93ca00640 1 -- 172.21.15.143:0/2014460704 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd930034630 con 0x7fd918004d00 2024-01-16T15:01:31.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.875+0000 7fd93ca00640 1 -- 172.21.15.143:0/2014460704 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93002f070 con 0x7fd918004d00 2024-01-16T15:01:31.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.875+0000 7fd93ca00640 1 -- 172.21.15.143:0/2014460704 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9300349f0 con 0x7fd918004d00 2024-01-16T15:01:31.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.876+0000 7fd93f200640 1 -- 172.21.15.143:0/2014460704 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd918004d00 msgr2=0x7fd918005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:31.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.876+0000 7fd93f200640 1 --2- 172.21.15.143:0/2014460704 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd918004d00 0x7fd918005180 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd93000a890 tx=0x7fd9180057f0 comp rx=0 tx=0).stop 2024-01-16T15:01:31.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.876+0000 7fd93e800640 1 -- 172.21.15.143:0/2014460704 reap_dead start 2024-01-16T15:01:31.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.876+0000 7fd93f200640 1 -- 172.21.15.143:0/2014460704 shutdown_connections 2024-01-16T15:01:31.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.876+0000 7fd93f200640 1 -- 172.21.15.143:0/2014460704 >> 172.21.15.143:0/2014460704 conn(0x7fd938061340 msgr2=0x7fd938061750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:31.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.876+0000 7fd93f200640 1 -- 172.21.15.143:0/2014460704 shutdown_connections 2024-01-16T15:01:31.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.876+0000 7fd93f200640 1 -- 172.21.15.143:0/2014460704 wait complete. 2024-01-16T15:01:31.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.878+0000 7fd93f200640 1 Processor -- start 2024-01-16T15:01:31.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.878+0000 7fd93f200640 1 -- start start 2024-01-16T15:01:31.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.879+0000 7fd93f200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd918004d00 0x7fd9380cb670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:31.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.879+0000 7fd93f200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd938153750 0x7fd9380cc870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:31.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.879+0000 7fd93f200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9380ccdb0 0x7fd93815b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:31.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.879+0000 7fd93f200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9380ca2f0 con 0x7fd9380ccdb0 2024-01-16T15:01:31.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.879+0000 7fd93f200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9380ca5d0 con 0x7fd918004d00 2024-01-16T15:01:31.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.879+0000 7fd93f200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9380ca8b0 con 0x7fd938153750 2024-01-16T15:01:31.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.879+0000 7fd93e800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9380ccdb0 0x7fd93815b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:31.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.879+0000 7fd93e800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9380ccdb0 0x7fd93815b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41086/0 (socket says 172.21.15.143:41086) 2024-01-16T15:01:31.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.879+0000 7fd93e800640 1 -- 172.21.15.143:0/2478077996 learned_addr learned my addr 172.21.15.143:0/2478077996 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:31.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.879+0000 7fd93de00640 1 --2- 172.21.15.143:0/2478077996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd918004d00 0x7fd9380cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:31.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.880+0000 7fd93e800640 1 -- 172.21.15.143:0/2478077996 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd938153750 msgr2=0x7fd9380cc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:31.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.880+0000 7fd93d400640 1 --2- 172.21.15.143:0/2478077996 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd938153750 0x7fd9380cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:31.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.880+0000 7fd93e800640 1 --2- 172.21.15.143:0/2478077996 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd938153750 0x7fd9380cc870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:31.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.880+0000 7fd93e800640 1 -- 172.21.15.143:0/2478077996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd918004d00 msgr2=0x7fd9380cb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:31.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.880+0000 7fd93e800640 1 --2- 172.21.15.143:0/2478077996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd918004d00 0x7fd9380cb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:31.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.880+0000 7fd93e800640 1 -- 172.21.15.143:0/2478077996 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9380cab90 con 0x7fd9380ccdb0 2024-01-16T15:01:31.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.880+0000 7fd93de00640 1 --2- 172.21.15.143:0/2478077996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd918004d00 0x7fd9380cb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:01:31.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.880+0000 7fd93e800640 1 --2- 172.21.15.143:0/2478077996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9380ccdb0 0x7fd93815b020 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd93000a890 tx=0x7fd930001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:31.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.880+0000 7fd936000640 1 -- 172.21.15.143:0/2478077996 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd930034840 con 0x7fd9380ccdb0 2024-01-16T15:01:31.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.880+0000 7fd936000640 1 -- 172.21.15.143:0/2478077996 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9300349a0 con 0x7fd9380ccdb0 2024-01-16T15:01:31.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.880+0000 7fd936000640 1 -- 172.21.15.143:0/2478077996 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd930034d10 con 0x7fd9380ccdb0 2024-01-16T15:01:31.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.880+0000 7fd93f200640 1 -- 172.21.15.143:0/2478077996 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9380cae70 con 0x7fd9380ccdb0 2024-01-16T15:01:31.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.880+0000 7fd93f200640 1 -- 172.21.15.143:0/2478077996 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd938153bb0 con 0x7fd9380ccdb0 2024-01-16T15:01:31.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.881+0000 7fd93f200640 1 -- 172.21.15.143:0/2478077996 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd938067990 con 0x7fd9380ccdb0 2024-01-16T15:01:31.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.882+0000 7fd936000640 1 -- 172.21.15.143:0/2478077996 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fd930031030 con 0x7fd9380ccdb0 2024-01-16T15:01:31.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.883+0000 7fd936000640 1 --2- 172.21.15.143:0/2478077996 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd91006b2f0 0x7fd91006d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:31.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.883+0000 7fd936000640 1 -- 172.21.15.143:0/2478077996 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd9300abd80 con 0x7fd9380ccdb0 2024-01-16T15:01:31.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.883+0000 7fd93de00640 1 --2- 172.21.15.143:0/2478077996 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd91006b2f0 0x7fd91006d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:31.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.884+0000 7fd93de00640 1 --2- 172.21.15.143:0/2478077996 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd91006b2f0 0x7fd91006d7b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd928009a00 tx=0x7fd928008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:31.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:31.888+0000 7fd936000640 1 -- 172.21.15.143:0/2478077996 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd930076be0 con 0x7fd9380ccdb0 2024-01-16T15:01:32.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.106+0000 7fd93f200640 1 -- 172.21.15.143:0/2478077996 --> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9380b9100 con 0x7fd91006b2f0 2024-01-16T15:01:32.099 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T15:01:32.116+0000 7f2770000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T15:01:32.099 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T15:01:32.116+0000 7f2770000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T15:01:32.099 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T15:01:32.116+0000 7f2770000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T15:01:32.099 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T15:01:32.117+0000 7f2770000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T15:01:32.099 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T15:01:32.117+0000 7f2770000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T15:01:32.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.163+0000 7fd936000640 1 -- 172.21.15.143:0/2478077996 <== mgr.4113 v2:172.21.15.143:6832/3153492115 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd928009670 con 0x7fd91006b2f0 2024-01-16T15:01:32.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.165+0000 7fd93f200640 1 -- 172.21.15.143:0/2478077996 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd91006b2f0 msgr2=0x7fd91006d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:32.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.165+0000 7fd93f200640 1 --2- 172.21.15.143:0/2478077996 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd91006b2f0 0x7fd91006d7b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd928009a00 tx=0x7fd928008040 comp rx=0 tx=0).stop 2024-01-16T15:01:32.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.165+0000 7fd93f200640 1 -- 172.21.15.143:0/2478077996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9380ccdb0 msgr2=0x7fd93815b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:32.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.165+0000 7fd93f200640 1 --2- 172.21.15.143:0/2478077996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9380ccdb0 0x7fd93815b020 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd93000a890 tx=0x7fd930001d80 comp rx=0 tx=0).stop 2024-01-16T15:01:32.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.166+0000 7fd93f200640 1 -- 172.21.15.143:0/2478077996 shutdown_connections 2024-01-16T15:01:32.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.166+0000 7fd93f200640 1 --2- 172.21.15.143:0/2478077996 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fd91006b2f0 0x7fd91006d7b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:32.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.166+0000 7fd93f200640 1 --2- 172.21.15.143:0/2478077996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9380ccdb0 0x7fd93815b020 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:32.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.166+0000 7fd93f200640 1 --2- 172.21.15.143:0/2478077996 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd938153750 0x7fd9380cc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:32.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.166+0000 7fd93f200640 1 --2- 172.21.15.143:0/2478077996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd918004d00 0x7fd9380cb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:32.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.166+0000 7fd93f200640 1 -- 172.21.15.143:0/2478077996 >> 172.21.15.143:0/2478077996 conn(0x7fd938061340 msgr2=0x7fd9380d3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:32.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.166+0000 7fd93f200640 1 -- 172.21.15.143:0/2478077996 shutdown_connections 2024-01-16T15:01:32.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.166+0000 7fd93f200640 1 -- 172.21.15.143:0/2478077996 wait complete. 2024-01-16T15:01:32.166 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-01-16T15:01:32.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.379+0000 7f0b0ca00640 1 Processor -- start 2024-01-16T15:01:32.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.380+0000 7f0b0ca00640 1 -- start start 2024-01-16T15:01:32.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.381+0000 7f0b0ca00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0b04072160 0x7f0b0415e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:32.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.381+0000 7f0b0ca00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0b0406de10 0x7f0b040705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:32.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.381+0000 7f0b0ca00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b0415ec90 con 0x7f0b040c49f0 2024-01-16T15:01:32.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.381+0000 7f0b0ca00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b040cd530 con 0x7f0b0406de10 2024-01-16T15:01:32.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.381+0000 7f0b0ca00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b040c4ed0 con 0x7f0b04072160 2024-01-16T15:01:32.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.382+0000 7f0b09600640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f0b040c49f0 0x7f0b0406d700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:58544/0 (socket says 172.21.15.143:58544) 2024-01-16T15:01:32.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.382+0000 7f0b09600640 1 -- 172.21.15.143:0/3340665443 learned_addr learned my addr 172.21.15.143:0/3340665443 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:01:32.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.382+0000 7f0b0aa00640 1 --2- 172.21.15.143:0/3340665443 >> v2:172.21.15.143:3300/0 conn(0x7f0b0406de10 0x7f0b040705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:32.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.382+0000 7f0b0a000640 1 --2- 172.21.15.143:0/3340665443 >> v2:172.21.15.204:3300/0 conn(0x7f0b04072160 0x7f0b0415e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:32.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.382+0000 7f0b08c00640 1 -- 172.21.15.143:0/3340665443 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 592586226 0 0) 0x7f0aec001110 con 0x7f0b040c49f0 2024-01-16T15:01:32.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.382+0000 7f0b08c00640 1 -- 172.21.15.143:0/3340665443 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ae4005730 con 0x7f0b040c49f0 2024-01-16T15:01:32.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.382+0000 7f0b0aa00640 1 -- 172.21.15.143:0/3340665443 >> v2:172.21.15.204:3300/0 conn(0x7f0b04072160 msgr2=0x7f0b0415e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:32.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.382+0000 7f0b0aa00640 1 --2- 172.21.15.143:0/3340665443 >> v2:172.21.15.204:3300/0 conn(0x7f0b04072160 0x7f0b0415e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:32.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.382+0000 7f0b0aa00640 1 -- 172.21.15.143:0/3340665443 >> v1:172.21.15.143:6789/0 conn(0x7f0b040c49f0 legacy=0x7f0b0406d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:32.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.382+0000 7f0b0aa00640 1 -- 172.21.15.143:0/3340665443 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b040c51b0 con 0x7f0b0406de10 2024-01-16T15:01:32.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.383+0000 7f0b0aa00640 1 --2- 172.21.15.143:0/3340665443 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b0406de10 0x7f0b040705c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0b040ccae0 tx=0x7f0afc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2230d6fee084bc server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:32.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.384+0000 7f0b08c00640 1 -- 172.21.15.143:0/3340665443 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0afc02cc80 con 0x7f0b0406de10 2024-01-16T15:01:32.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.384+0000 7f0b08c00640 1 -- 172.21.15.143:0/3340665443 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0afc02cde0 con 0x7f0b0406de10 2024-01-16T15:01:32.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.384+0000 7f0b08c00640 1 -- 172.21.15.143:0/3340665443 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0afc034660 con 0x7f0b0406de10 2024-01-16T15:01:32.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.384+0000 7f0b0ca00640 1 -- 172.21.15.143:0/3340665443 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b0406de10 msgr2=0x7f0b040705c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:32.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.384+0000 7f0b0ca00640 1 --2- 172.21.15.143:0/3340665443 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b0406de10 0x7f0b040705c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0b040ccae0 tx=0x7f0afc02bed0 comp rx=0 tx=0).stop 2024-01-16T15:01:32.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.384+0000 7f0b0ca00640 1 -- 172.21.15.143:0/3340665443 shutdown_connections 2024-01-16T15:01:32.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.384+0000 7f0b0ca00640 1 --2- 172.21.15.143:0/3340665443 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b0406de10 0x7f0b040705c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:32.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.384+0000 7f0b0ca00640 1 --2- 172.21.15.143:0/3340665443 >> v2:172.21.15.204:3300/0 conn(0x7f0b04072160 0x7f0b0415e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:32.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.384+0000 7f0b0ca00640 1 -- 172.21.15.143:0/3340665443 >> 172.21.15.143:0/3340665443 conn(0x7f0b040b0c50 msgr2=0x7f0b040b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:32.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.384+0000 7f0b0ca00640 1 -- 172.21.15.143:0/3340665443 shutdown_connections 2024-01-16T15:01:32.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.385+0000 7f0b0ca00640 1 -- 172.21.15.143:0/3340665443 wait complete. 2024-01-16T15:01:32.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.386+0000 7f0b0ca00640 1 Processor -- start 2024-01-16T15:01:32.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.386+0000 7f0b0ca00640 1 -- start start 2024-01-16T15:01:32.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.387+0000 7f0b0ca00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b0406de10 0x7f0b04158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:32.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.387+0000 7f0b0ca00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b04072160 0x7f0b040b3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:32.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.387+0000 7f0b0ca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b040c49f0 0x7f0b040cb290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:32.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.387+0000 7f0b0ca00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b04158a70 con 0x7f0b0406de10 2024-01-16T15:01:32.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.387+0000 7f0b0ca00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b040b3f90 con 0x7f0b040c49f0 2024-01-16T15:01:32.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.387+0000 7f0b0ca00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b040b4270 con 0x7f0b04072160 2024-01-16T15:01:32.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.387+0000 7f0b0a000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b0406de10 0x7f0b04158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:32.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.388+0000 7f0b0a000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b0406de10 0x7f0b04158530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41116/0 (socket says 172.21.15.143:41116) 2024-01-16T15:01:32.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.388+0000 7f0b0a000640 1 -- 172.21.15.143:0/246082629 learned_addr learned my addr 172.21.15.143:0/246082629 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:32.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.388+0000 7f0b09600640 1 --2- 172.21.15.143:0/246082629 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b04072160 0x7f0b040b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:32.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.388+0000 7f0b0aa00640 1 --2- 172.21.15.143:0/246082629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b040c49f0 0x7f0b040cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:32.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.388+0000 7f0b0a000640 1 -- 172.21.15.143:0/246082629 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b04072160 msgr2=0x7f0b040b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:32.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.388+0000 7f0b0a000640 1 --2- 172.21.15.143:0/246082629 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b04072160 0x7f0b040b3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:32.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.388+0000 7f0b0a000640 1 -- 172.21.15.143:0/246082629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b040c49f0 msgr2=0x7f0b040cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:32.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.388+0000 7f0b0a000640 1 --2- 172.21.15.143:0/246082629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b040c49f0 0x7f0b040cb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:32.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.388+0000 7f0b0a000640 1 -- 172.21.15.143:0/246082629 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0afc009a70 con 0x7f0b0406de10 2024-01-16T15:01:32.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.388+0000 7f0b0aa00640 1 --2- 172.21.15.143:0/246082629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b040c49f0 0x7f0b040cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:01:32.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.388+0000 7f0b0a000640 1 --2- 172.21.15.143:0/246082629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b0406de10 0x7f0b04158530 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f0af400d800 tx=0x7f0af400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:32.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.389+0000 7f0b02000640 1 -- 172.21.15.143:0/246082629 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af400b6c0 con 0x7f0b0406de10 2024-01-16T15:01:32.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.389+0000 7f0b02000640 1 -- 172.21.15.143:0/246082629 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0af400b820 con 0x7f0b0406de10 2024-01-16T15:01:32.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.389+0000 7f0b02000640 1 -- 172.21.15.143:0/246082629 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af400bbb0 con 0x7f0b0406de10 2024-01-16T15:01:32.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.389+0000 7f0b0ca00640 1 -- 172.21.15.143:0/246082629 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b040b4550 con 0x7f0b0406de10 2024-01-16T15:01:32.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.389+0000 7f0b0ca00640 1 -- 172.21.15.143:0/246082629 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b040b4b10 con 0x7f0b0406de10 2024-01-16T15:01:32.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.390+0000 7f0b0ca00640 1 -- 172.21.15.143:0/246082629 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b040c3ec0 con 0x7f0b0406de10 2024-01-16T15:01:32.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.391+0000 7f0b02000640 1 -- 172.21.15.143:0/246082629 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f0af400bd10 con 0x7f0b0406de10 2024-01-16T15:01:32.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.392+0000 7f0b02000640 1 --2- 172.21.15.143:0/246082629 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0adc06b2c0 0x7f0adc06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:32.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.392+0000 7f0b02000640 1 -- 172.21.15.143:0/246082629 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0af408a210 con 0x7f0b0406de10 2024-01-16T15:01:32.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.392+0000 7f0b09600640 1 --2- 172.21.15.143:0/246082629 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0adc06b2c0 0x7f0adc06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:32.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.393+0000 7f0b09600640 1 --2- 172.21.15.143:0/246082629 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0adc06b2c0 0x7f0adc06d780 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0b041f69c0 tx=0x7f0aec009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:32.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.396+0000 7f0b02000640 1 -- 172.21.15.143:0/246082629 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0af4054fe0 con 0x7f0b0406de10 2024-01-16T15:01:32.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.619+0000 7f0b0ca00640 1 -- 172.21.15.143:0/246082629 --> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b040aea00 con 0x7f0adc06b2c0 2024-01-16T15:01:32.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.625+0000 7f0b02000640 1 -- 172.21.15.143:0/246082629 <== mgr.4113 v2:172.21.15.143:6832/3153492115 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f0b040aea00 con 0x7f0adc06b2c0 2024-01-16T15:01:32.606 INFO:teuthology.orchestra.run.smithi143.stdout:/volumes/_nogroup/sv1/35696085-26f8-4228-885d-3e94e8d48712 2024-01-16T15:01:32.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.628+0000 7f0b0ca00640 1 -- 172.21.15.143:0/246082629 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0adc06b2c0 msgr2=0x7f0adc06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:32.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.628+0000 7f0b0ca00640 1 --2- 172.21.15.143:0/246082629 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0adc06b2c0 0x7f0adc06d780 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0b041f69c0 tx=0x7f0aec009040 comp rx=0 tx=0).stop 2024-01-16T15:01:32.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.628+0000 7f0b0ca00640 1 -- 172.21.15.143:0/246082629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b0406de10 msgr2=0x7f0b04158530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:32.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.628+0000 7f0b0ca00640 1 --2- 172.21.15.143:0/246082629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b0406de10 0x7f0b04158530 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f0af400d800 tx=0x7f0af400dcc0 comp rx=0 tx=0).stop 2024-01-16T15:01:32.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.628+0000 7f0b0ca00640 1 -- 172.21.15.143:0/246082629 shutdown_connections 2024-01-16T15:01:32.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.628+0000 7f0b0ca00640 1 --2- 172.21.15.143:0/246082629 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0adc06b2c0 0x7f0adc06d780 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:32.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.628+0000 7f0b0ca00640 1 --2- 172.21.15.143:0/246082629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b040c49f0 0x7f0b040cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:32.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.628+0000 7f0b0ca00640 1 --2- 172.21.15.143:0/246082629 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b04072160 0x7f0b040b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:32.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.628+0000 7f0b0ca00640 1 --2- 172.21.15.143:0/246082629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b0406de10 0x7f0b04158530 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:32.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.628+0000 7f0b0ca00640 1 -- 172.21.15.143:0/246082629 >> 172.21.15.143:0/246082629 conn(0x7f0b040b0c50 msgr2=0x7f0b040ae210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:32.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.628+0000 7f0b0ca00640 1 -- 172.21.15.143:0/246082629 shutdown_connections 2024-01-16T15:01:32.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:32.628+0000 7f0b0ca00640 1 -- 172.21.15.143:0/246082629 wait complete. 2024-01-16T15:01:32.626 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/35696085-26f8-4228-885d-3e94e8d48712 2024-01-16T15:01:32.626 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T15:01:32.627 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T15:01:32.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:01:32.675 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/35696085-26f8-4228-885d-3e94e8d48712/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-16T15:01:40.267 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records in 2024-01-16T15:01:40.267 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records out 2024-01-16T15:01:40.267 INFO:teuthology.orchestra.run.smithi143.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.48508 s, 143 MB/s 2024-01-16T15:01:40.268 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T15:01:40.268 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T15:01:40.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:01:40.304 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/35696085-26f8-4228-885d-3e94e8d48712/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-16T15:01:47.969 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records in 2024-01-16T15:01:47.970 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records out 2024-01-16T15:01:47.970 INFO:teuthology.orchestra.run.smithi143.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.57766 s, 142 MB/s 2024-01-16T15:01:47.971 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T15:01:47.971 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T15:01:48.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:01:48.010 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/35696085-26f8-4228-885d-3e94e8d48712/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-16T15:01:53.409 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:01:53.420 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:01:55.865 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records in 2024-01-16T15:01:55.865 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records out 2024-01-16T15:01:55.865 INFO:teuthology.orchestra.run.smithi143.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.76784 s, 138 MB/s 2024-01-16T15:01:55.866 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/clone_delay_every_regfile 2 2024-01-16T15:01:56.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.062+0000 7f7106c00640 1 Processor -- start 2024-01-16T15:01:56.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.063+0000 7f7106c00640 1 -- start start 2024-01-16T15:01:56.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.064+0000 7f7106c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f710006de10 0x7f710015e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:56.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.064+0000 7f7106c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f710015ec90 con 0x7f71000c49f0 2024-01-16T15:01:56.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.064+0000 7f7106c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71000cd530 con 0x7f7100072160 2024-01-16T15:01:56.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.064+0000 7f7106c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71000c4ed0 con 0x7f710006de10 2024-01-16T15:01:56.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.065+0000 7f7104c00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f71000c49f0 0x7f71000705c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51928/0 (socket says 172.21.15.143:51928) 2024-01-16T15:01:56.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.065+0000 7f7104c00640 1 -- 172.21.15.143:0/2158159166 learned_addr learned my addr 172.21.15.143:0/2158159166 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:01:56.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.065+0000 7f70f7200640 1 --2- 172.21.15.143:0/2158159166 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f710006de10 0x7f710015e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:56.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.065+0000 7f70ff400640 1 -- 172.21.15.143:0/2158159166 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3417203554 0 0) 0x7f70f8001110 con 0x7f71000c49f0 2024-01-16T15:01:56.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.065+0000 7f70ff400640 1 -- 172.21.15.143:0/2158159166 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70e0003520 con 0x7f71000c49f0 2024-01-16T15:01:56.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.065+0000 7f70ff400640 1 -- 172.21.15.143:0/2158159166 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1374983433 0 0) 0x7f70e0003520 con 0x7f71000c49f0 2024-01-16T15:01:56.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.065+0000 7f70ff400640 1 -- 172.21.15.143:0/2158159166 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f710006de10 msgr2=0x7f710015e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:56.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.065+0000 7f70ff400640 1 --2- 172.21.15.143:0/2158159166 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f710006de10 0x7f710015e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:56.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.065+0000 7f70ff400640 1 -- 172.21.15.143:0/2158159166 >> v1:172.21.15.204:6789/0 conn(0x7f7100072160 legacy=0x7f710006d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:56.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.065+0000 7f70ff400640 1 -- 172.21.15.143:0/2158159166 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71000c51b0 con 0x7f71000c49f0 2024-01-16T15:01:56.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.065+0000 7f70ff400640 1 -- 172.21.15.143:0/2158159166 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7f70f8001800 con 0x7f71000c49f0 2024-01-16T15:01:56.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.066+0000 7f70ff400640 1 -- 172.21.15.143:0/2158159166 >> v1:172.21.15.143:6789/0 conn(0x7f71000c49f0 legacy=0x7f71000705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:56.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.066+0000 7f70ff400640 1 --2- 172.21.15.143:0/2158159166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70e00039a0 0x7f70e0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:56.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.066+0000 7f70ff400640 1 --2- 172.21.15.143:0/2158159166 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f710006de10 0x7f70e00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:56.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.066+0000 7f70ff400640 1 --2- 172.21.15.143:0/2158159166 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70e0004d00 0x7f70e0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:56.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.066+0000 7f70ff400640 1 -- 172.21.15.143:0/2158159166 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71000c4ed0 con 0x7f70e0004d00 2024-01-16T15:01:56.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.066+0000 7f70ff400640 1 -- 172.21.15.143:0/2158159166 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e000bb60 con 0x7f70e00039a0 2024-01-16T15:01:56.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.066+0000 7f70ff400640 1 -- 172.21.15.143:0/2158159166 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e000df30 con 0x7f710006de10 2024-01-16T15:01:56.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.066+0000 7f7104c00640 1 --2- 172.21.15.143:0/2158159166 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70e0004d00 0x7f70e0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:56.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.066+0000 7f70ffe00640 1 --2- 172.21.15.143:0/2158159166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70e00039a0 0x7f70e0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:56.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.066+0000 7f7104c00640 1 -- 172.21.15.143:0/2158159166 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f710006de10 msgr2=0x7f70e00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:56.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.066+0000 7f7104c00640 1 --2- 172.21.15.143:0/2158159166 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f710006de10 0x7f70e00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:56.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.066+0000 7f7104c00640 1 -- 172.21.15.143:0/2158159166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70e00039a0 msgr2=0x7f70e0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:56.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.066+0000 7f7104c00640 1 --2- 172.21.15.143:0/2158159166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70e00039a0 0x7f70e0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:56.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.066+0000 7f7104c00640 1 -- 172.21.15.143:0/2158159166 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e0003520 con 0x7f70e0004d00 2024-01-16T15:01:56.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.066+0000 7f70ffe00640 1 --2- 172.21.15.143:0/2158159166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70e00039a0 0x7f70e0003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:01:56.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.067+0000 7f7104c00640 1 --2- 172.21.15.143:0/2158159166 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70e0004d00 0x7f70e0005180 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f71000c4e00 tx=0x7f70e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be1cecd7c191cea3 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:56.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.068+0000 7f70ff400640 1 -- 172.21.15.143:0/2158159166 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f8005ad0 con 0x7f70e0004d00 2024-01-16T15:01:56.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.068+0000 7f70ff400640 1 -- 172.21.15.143:0/2158159166 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f8005c30 con 0x7f70e0004d00 2024-01-16T15:01:56.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.068+0000 7f70ff400640 1 -- 172.21.15.143:0/2158159166 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f802c770 con 0x7f70e0004d00 2024-01-16T15:01:56.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.068+0000 7f7106c00640 1 -- 172.21.15.143:0/2158159166 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70e0004d00 msgr2=0x7f70e0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:56.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.068+0000 7f7106c00640 1 --2- 172.21.15.143:0/2158159166 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70e0004d00 0x7f70e0005180 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f71000c4e00 tx=0x7f70e00057f0 comp rx=0 tx=0).stop 2024-01-16T15:01:56.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.069+0000 7f7104c00640 1 -- 172.21.15.143:0/2158159166 reap_dead start 2024-01-16T15:01:56.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.069+0000 7f7106c00640 1 -- 172.21.15.143:0/2158159166 shutdown_connections 2024-01-16T15:01:56.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.069+0000 7f7106c00640 1 -- 172.21.15.143:0/2158159166 >> 172.21.15.143:0/2158159166 conn(0x7f71000b0c50 msgr2=0x7f71000b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:56.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.069+0000 7f7106c00640 1 -- 172.21.15.143:0/2158159166 shutdown_connections 2024-01-16T15:01:56.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.069+0000 7f7106c00640 1 -- 172.21.15.143:0/2158159166 wait complete. 2024-01-16T15:01:56.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.071+0000 7f7106c00640 1 Processor -- start 2024-01-16T15:01:56.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.071+0000 7f7106c00640 1 -- start start 2024-01-16T15:01:56.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.072+0000 7f7106c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70e0004d00 0x7f7100159760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:56.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.072+0000 7f7106c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f710015b770 0x7f7100159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:56.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.072+0000 7f7106c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71001ef000 0x7f71001ef480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:56.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.072+0000 7f7106c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710015bbf0 con 0x7f70e0004d00 2024-01-16T15:01:56.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.072+0000 7f7106c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71000b3b60 con 0x7f71001ef000 2024-01-16T15:01:56.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.072+0000 7f7106c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71000b2f50 con 0x7f710015b770 2024-01-16T15:01:56.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.072+0000 7f70ffe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70e0004d00 0x7f7100159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:56.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.072+0000 7f70ffe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70e0004d00 0x7f7100159760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39384/0 (socket says 172.21.15.143:39384) 2024-01-16T15:01:56.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.072+0000 7f70ffe00640 1 -- 172.21.15.143:0/3617128995 learned_addr learned my addr 172.21.15.143:0/3617128995 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:56.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.073+0000 7f70f7200640 1 --2- 172.21.15.143:0/3617128995 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f710015b770 0x7f7100159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:56.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.073+0000 7f70ffe00640 1 -- 172.21.15.143:0/3617128995 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f710015b770 msgr2=0x7f7100159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:56.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.073+0000 7f70ffe00640 1 --2- 172.21.15.143:0/3617128995 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f710015b770 0x7f7100159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:56.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.073+0000 7f70ffe00640 1 -- 172.21.15.143:0/3617128995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71001ef000 msgr2=0x7f71001ef480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:56.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.073+0000 7f70ffe00640 1 --2- 172.21.15.143:0/3617128995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71001ef000 0x7f71001ef480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:56.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.073+0000 7f70ffe00640 1 -- 172.21.15.143:0/3617128995 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71000b3230 con 0x7f70e0004d00 2024-01-16T15:01:56.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.073+0000 7f70f7200640 1 --2- 172.21.15.143:0/3617128995 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f710015b770 0x7f7100159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:01:56.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.073+0000 7f70ffe00640 1 --2- 172.21.15.143:0/3617128995 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70e0004d00 0x7f7100159760 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f70f000db10 tx=0x7f70f0002080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:56.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.073+0000 7f70fcc00640 1 -- 172.21.15.143:0/3617128995 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f0003da0 con 0x7f70e0004d00 2024-01-16T15:01:56.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.073+0000 7f70fcc00640 1 -- 172.21.15.143:0/3617128995 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f000b740 con 0x7f70e0004d00 2024-01-16T15:01:56.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.073+0000 7f70fcc00640 1 -- 172.21.15.143:0/3617128995 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f000baf0 con 0x7f70e0004d00 2024-01-16T15:01:56.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.073+0000 7f7106c00640 1 -- 172.21.15.143:0/3617128995 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71000b3510 con 0x7f70e0004d00 2024-01-16T15:01:56.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.073+0000 7f7106c00640 1 -- 172.21.15.143:0/3617128995 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7100159f60 con 0x7f70e0004d00 2024-01-16T15:01:56.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.075+0000 7f7106c00640 1 -- 172.21.15.143:0/3617128995 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f710004e8b0 con 0x7f70e0004d00 2024-01-16T15:01:56.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.075+0000 7f70fcc00640 1 -- 172.21.15.143:0/3617128995 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 6) v1 ==== 89590+0+0 (secure 0 0 0) 0x7f70f000bc50 con 0x7f70e0004d00 2024-01-16T15:01:56.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.077+0000 7f70fcc00640 1 --2- 172.21.15.143:0/3617128995 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f70d806b350 0x7f70d806d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:56.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.077+0000 7f70fcc00640 1 -- 172.21.15.143:0/3617128995 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f70f0089f90 con 0x7f70e0004d00 2024-01-16T15:01:56.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.077+0000 7f70f7200640 1 --2- 172.21.15.143:0/3617128995 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f70d806b350 0x7f70d806d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:56.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.077+0000 7f70f7200640 1 --2- 172.21.15.143:0/3617128995 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f70d806b350 0x7f70d806d810 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f70e80047b0 tx=0x7f70e8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:56.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.081+0000 7f70fcc00640 1 -- 172.21.15.143:0/3617128995 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f70f0054d90 con 0x7f70e0004d00 2024-01-16T15:01:56.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.255+0000 7f7106c00640 1 -- 172.21.15.143:0/3617128995 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/clone_delay_every_regfile}] v 0) v1 -- 0x7f710015dab0 con 0x7f70e0004d00 2024-01-16T15:01:56.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.259+0000 7f70fcc00640 1 -- 172.21.15.143:0/3617128995 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/clone_delay_every_regfile}]=0 v11)=0 v11) v1 ==== 135+0+0 (secure 0 0 0) 0x7f70f00583b0 con 0x7f70e0004d00 2024-01-16T15:01:56.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.262+0000 7f7106c00640 1 -- 172.21.15.143:0/3617128995 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f70d806b350 msgr2=0x7f70d806d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:56.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.262+0000 7f7106c00640 1 --2- 172.21.15.143:0/3617128995 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f70d806b350 0x7f70d806d810 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f70e80047b0 tx=0x7f70e8008040 comp rx=0 tx=0).stop 2024-01-16T15:01:56.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.262+0000 7f7106c00640 1 -- 172.21.15.143:0/3617128995 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70e0004d00 msgr2=0x7f7100159760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:56.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.262+0000 7f7106c00640 1 --2- 172.21.15.143:0/3617128995 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70e0004d00 0x7f7100159760 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f70f000db10 tx=0x7f70f0002080 comp rx=0 tx=0).stop 2024-01-16T15:01:56.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.263+0000 7f7106c00640 1 -- 172.21.15.143:0/3617128995 shutdown_connections 2024-01-16T15:01:56.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.263+0000 7f7106c00640 1 --2- 172.21.15.143:0/3617128995 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f70d806b350 0x7f70d806d810 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:56.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.263+0000 7f7106c00640 1 --2- 172.21.15.143:0/3617128995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71001ef000 0x7f71001ef480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:56.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.263+0000 7f7106c00640 1 --2- 172.21.15.143:0/3617128995 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f710015b770 0x7f7100159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:56.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.263+0000 7f7106c00640 1 --2- 172.21.15.143:0/3617128995 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70e0004d00 0x7f7100159760 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:56.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.263+0000 7f7106c00640 1 -- 172.21.15.143:0/3617128995 >> 172.21.15.143:0/3617128995 conn(0x7f71000b0c50 msgr2=0x7f71000704c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:56.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.263+0000 7f7106c00640 1 -- 172.21.15.143:0/3617128995 shutdown_connections 2024-01-16T15:01:56.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.263+0000 7f7106c00640 1 -- 172.21.15.143:0/3617128995 wait complete. 2024-01-16T15:01:56.259 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-01-16T15:01:56.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.479+0000 7efe41000640 1 Processor -- start 2024-01-16T15:01:56.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.479+0000 7efe41000640 1 -- start start 2024-01-16T15:01:56.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.480+0000 7efe41000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7efe3c15e390 0x7efe3c161df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:56.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.480+0000 7efe41000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe3c0ce0b0 con 0x7efe3c157c70 2024-01-16T15:01:56.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.480+0000 7efe41000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe3c161960 con 0x7efe3c0cfb60 2024-01-16T15:01:56.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.480+0000 7efe41000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe3c15e8d0 con 0x7efe3c15e390 2024-01-16T15:01:56.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.481+0000 7efe3a800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7efe3c15e390 0x7efe3c161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:56.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.481+0000 7efe39e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7efe3c157c70 0x7efe3c0cf450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51932/0 (socket says 172.21.15.143:51932) 2024-01-16T15:01:56.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.481+0000 7efe3a800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7efe3c15e390 0x7efe3c161df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39396/0 (socket says 172.21.15.143:39396) 2024-01-16T15:01:56.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.481+0000 7efe39e00640 1 -- 172.21.15.143:0/1646901002 learned_addr learned my addr 172.21.15.143:0/1646901002 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:01:56.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.481+0000 7efe3a800640 1 -- 172.21.15.143:0/1646901002 >> v1:172.21.15.204:6789/0 conn(0x7efe3c0cfb60 legacy=0x7efe3c15dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:56.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.481+0000 7efe3a800640 1 -- 172.21.15.143:0/1646901002 >> v1:172.21.15.143:6789/0 conn(0x7efe3c157c70 legacy=0x7efe3c0cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:56.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.481+0000 7efe3a800640 1 -- 172.21.15.143:0/1646901002 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe3c15ebb0 con 0x7efe3c15e390 2024-01-16T15:01:56.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.481+0000 7efe39400640 1 -- 172.21.15.143:0/1646901002 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2338718412 0 0) 0x7efe3c0ce0b0 con 0x7efe3c157c70 2024-01-16T15:01:56.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.482+0000 7efe3a800640 1 --2- 172.21.15.143:0/1646901002 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe3c15e390 0x7efe3c161df0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7efe34009d90 tx=0x7efe3402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4460c8f74b0dcae server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:56.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.483+0000 7efe39400640 1 -- 172.21.15.143:0/1646901002 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3402cbd0 con 0x7efe3c15e390 2024-01-16T15:01:56.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.483+0000 7efe39400640 1 -- 172.21.15.143:0/1646901002 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe3402cd30 con 0x7efe3c15e390 2024-01-16T15:01:56.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.483+0000 7efe39400640 1 -- 172.21.15.143:0/1646901002 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3402cea0 con 0x7efe3c15e390 2024-01-16T15:01:56.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.483+0000 7efe41000640 1 -- 172.21.15.143:0/1646901002 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe3c15e390 msgr2=0x7efe3c161df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:56.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.483+0000 7efe41000640 1 --2- 172.21.15.143:0/1646901002 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe3c15e390 0x7efe3c161df0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7efe34009d90 tx=0x7efe3402be20 comp rx=0 tx=0).stop 2024-01-16T15:01:56.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.483+0000 7efe41000640 1 -- 172.21.15.143:0/1646901002 shutdown_connections 2024-01-16T15:01:56.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.483+0000 7efe41000640 1 --2- 172.21.15.143:0/1646901002 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe3c15e390 0x7efe3c161df0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:56.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.483+0000 7efe41000640 1 -- 172.21.15.143:0/1646901002 >> 172.21.15.143:0/1646901002 conn(0x7efe3c06a920 msgr2=0x7efe3c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:56.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.483+0000 7efe41000640 1 -- 172.21.15.143:0/1646901002 shutdown_connections 2024-01-16T15:01:56.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.483+0000 7efe41000640 1 -- 172.21.15.143:0/1646901002 wait complete. 2024-01-16T15:01:56.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.485+0000 7efe41000640 1 Processor -- start 2024-01-16T15:01:56.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.485+0000 7efe41000640 1 -- start start 2024-01-16T15:01:56.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.486+0000 7efe41000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe3c0cfb60 0x7efe3c14d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:56.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.486+0000 7efe41000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe3c157c70 0x7efe3c09b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:56.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.486+0000 7efe41000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efe3c15e390 0x7efe3c0cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:56.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.486+0000 7efe41000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c09be40 con 0x7efe3c0cfb60 2024-01-16T15:01:56.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.486+0000 7efe41000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c09cc30 con 0x7efe3c157c70 2024-01-16T15:01:56.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.486+0000 7efe41000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c0ad7b0 con 0x7efe3c15e390 2024-01-16T15:01:56.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.486+0000 7efe39e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe3c0cfb60 0x7efe3c14d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:56.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.486+0000 7efe39e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe3c0cfb60 0x7efe3c14d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39404/0 (socket says 172.21.15.143:39404) 2024-01-16T15:01:56.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.486+0000 7efe39e00640 1 -- 172.21.15.143:0/3663394784 learned_addr learned my addr 172.21.15.143:0/3663394784 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:56.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.487+0000 7efe3a800640 1 --2- 172.21.15.143:0/3663394784 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efe3c15e390 0x7efe3c0cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:56.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.487+0000 7efe33e00640 1 --2- 172.21.15.143:0/3663394784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe3c157c70 0x7efe3c09b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:56.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.487+0000 7efe39e00640 1 -- 172.21.15.143:0/3663394784 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efe3c15e390 msgr2=0x7efe3c0cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:56.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.487+0000 7efe39e00640 1 --2- 172.21.15.143:0/3663394784 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efe3c15e390 0x7efe3c0cb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:56.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.487+0000 7efe39e00640 1 -- 172.21.15.143:0/3663394784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe3c157c70 msgr2=0x7efe3c09b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:56.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.487+0000 7efe39e00640 1 --2- 172.21.15.143:0/3663394784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe3c157c70 0x7efe3c09b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:56.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.487+0000 7efe39e00640 1 -- 172.21.15.143:0/3663394784 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe34009a70 con 0x7efe3c0cfb60 2024-01-16T15:01:56.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.487+0000 7efe33e00640 1 --2- 172.21.15.143:0/3663394784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe3c157c70 0x7efe3c09b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:01:56.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.487+0000 7efe39e00640 1 --2- 172.21.15.143:0/3663394784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe3c0cfb60 0x7efe3c14d060 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7efe2c00c4d0 tx=0x7efe2c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:56.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.487+0000 7efe32000640 1 -- 172.21.15.143:0/3663394784 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c0074e0 con 0x7efe3c0cfb60 2024-01-16T15:01:56.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.487+0000 7efe32000640 1 -- 172.21.15.143:0/3663394784 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2c007660 con 0x7efe3c0cfb60 2024-01-16T15:01:56.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.487+0000 7efe32000640 1 -- 172.21.15.143:0/3663394784 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c004a20 con 0x7efe3c0cfb60 2024-01-16T15:01:56.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.487+0000 7efe41000640 1 -- 172.21.15.143:0/3663394784 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe3c09d0c0 con 0x7efe3c0cfb60 2024-01-16T15:01:56.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.487+0000 7efe41000640 1 -- 172.21.15.143:0/3663394784 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe3c09c0d0 con 0x7efe3c0cfb60 2024-01-16T15:01:56.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.488+0000 7efe41000640 1 -- 172.21.15.143:0/3663394784 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe3c067470 con 0x7efe3c0cfb60 2024-01-16T15:01:56.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.489+0000 7efe32000640 1 -- 172.21.15.143:0/3663394784 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 6) v1 ==== 89590+0+0 (secure 0 0 0) 0x7efe2c004b80 con 0x7efe3c0cfb60 2024-01-16T15:01:56.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.490+0000 7efe32000640 1 --2- 172.21.15.143:0/3663394784 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7efe1406b380 0x7efe1406d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:56.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.490+0000 7efe32000640 1 -- 172.21.15.143:0/3663394784 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efe2c088da0 con 0x7efe3c0cfb60 2024-01-16T15:01:56.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.490+0000 7efe33e00640 1 --2- 172.21.15.143:0/3663394784 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7efe1406b380 0x7efe1406d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:56.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.491+0000 7efe33e00640 1 --2- 172.21.15.143:0/3663394784 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7efe1406b380 0x7efe1406d840 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efe3c15d880 tx=0x7efe24008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:56.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.495+0000 7efe32000640 1 -- 172.21.15.143:0/3663394784 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efe2c053c20 con 0x7efe3c0cfb60 2024-01-16T15:01:56.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.698+0000 7efe41000640 1 -- 172.21.15.143:0/3663394784 --> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) v1 -- 0x7efe3c158050 con 0x7efe1406b380 2024-01-16T15:01:56.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.730+0000 7efe32000640 1 -- 172.21.15.143:0/3663394784 <== mgr.4113 v2:172.21.15.143:6832/3153492115 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7efe2400ad80 con 0x7efe1406b380 2024-01-16T15:01:56.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.732+0000 7efe41000640 1 -- 172.21.15.143:0/3663394784 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7efe1406b380 msgr2=0x7efe1406d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:56.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.732+0000 7efe41000640 1 --2- 172.21.15.143:0/3663394784 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7efe1406b380 0x7efe1406d840 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efe3c15d880 tx=0x7efe24008040 comp rx=0 tx=0).stop 2024-01-16T15:01:56.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.732+0000 7efe41000640 1 -- 172.21.15.143:0/3663394784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe3c0cfb60 msgr2=0x7efe3c14d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:56.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.732+0000 7efe41000640 1 --2- 172.21.15.143:0/3663394784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe3c0cfb60 0x7efe3c14d060 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7efe2c00c4d0 tx=0x7efe2c00c990 comp rx=0 tx=0).stop 2024-01-16T15:01:56.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.732+0000 7efe41000640 1 -- 172.21.15.143:0/3663394784 shutdown_connections 2024-01-16T15:01:56.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.732+0000 7efe41000640 1 --2- 172.21.15.143:0/3663394784 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7efe1406b380 0x7efe1406d840 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:56.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.732+0000 7efe41000640 1 --2- 172.21.15.143:0/3663394784 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efe3c15e390 0x7efe3c0cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:56.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.732+0000 7efe41000640 1 --2- 172.21.15.143:0/3663394784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe3c157c70 0x7efe3c09b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:56.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.732+0000 7efe41000640 1 --2- 172.21.15.143:0/3663394784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe3c0cfb60 0x7efe3c14d060 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:56.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.732+0000 7efe41000640 1 -- 172.21.15.143:0/3663394784 >> 172.21.15.143:0/3663394784 conn(0x7efe3c06a920 msgr2=0x7efe3c15f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:56.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.732+0000 7efe41000640 1 -- 172.21.15.143:0/3663394784 shutdown_connections 2024-01-16T15:01:56.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.732+0000 7efe41000640 1 -- 172.21.15.143:0/3663394784 wait complete. 2024-01-16T15:01:56.728 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-01-16T15:01:56.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.936+0000 7fdef7200640 1 Processor -- start 2024-01-16T15:01:56.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.937+0000 7fdef7200640 1 -- start start 2024-01-16T15:01:56.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.938+0000 7fdef7200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdef815d310 0x7fdef80ca120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:56.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.938+0000 7fdef7200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdef80b0b40 0x7fdef80b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:56.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.938+0000 7fdef7200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdef8160f70 0x7fdef81613d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:56.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.938+0000 7fdef7200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdef806cd10 con 0x7fdef8160f70 2024-01-16T15:01:56.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.938+0000 7fdef7200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdef80b14a0 con 0x7fdef80b0b40 2024-01-16T15:01:56.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.938+0000 7fdef7200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef80ae5d0 con 0x7fdef815d310 2024-01-16T15:01:56.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.938+0000 7fdef6800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdef8160f70 0x7fdef81613d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:56.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.938+0000 7fdef6800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdef8160f70 0x7fdef81613d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39408/0 (socket says 172.21.15.143:39408) 2024-01-16T15:01:56.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.938+0000 7fdef6800640 1 -- 172.21.15.143:0/131738740 learned_addr learned my addr 172.21.15.143:0/131738740 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:56.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.939+0000 7fdeed200640 1 --2- 172.21.15.143:0/131738740 >> v2:172.21.15.204:3300/0 conn(0x7fdef80b0b40 0x7fdef80b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:56.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.939+0000 7fdef5e00640 1 --2- 172.21.15.143:0/131738740 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdef815d310 0x7fdef80ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:56.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.939+0000 7fdef6800640 1 -- 172.21.15.143:0/131738740 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdef815d310 msgr2=0x7fdef80ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:56.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.939+0000 7fdef6800640 1 --2- 172.21.15.143:0/131738740 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdef815d310 0x7fdef80ca120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:56.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.939+0000 7fdef6800640 1 -- 172.21.15.143:0/131738740 >> v2:172.21.15.204:3300/0 conn(0x7fdef80b0b40 msgr2=0x7fdef80b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:01:56.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.939+0000 7fdef6800640 1 --2- 172.21.15.143:0/131738740 >> v2:172.21.15.204:3300/0 conn(0x7fdef80b0b40 0x7fdef80b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:56.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.939+0000 7fdef6800640 1 -- 172.21.15.143:0/131738740 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef80ae8b0 con 0x7fdef8160f70 2024-01-16T15:01:56.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.940+0000 7fdef6800640 1 --2- 172.21.15.143:0/131738740 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdef8160f70 0x7fdef81613d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdef80ccb30 tx=0x7fdef002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8633fe0766075839 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:56.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.940+0000 7fdef5400640 1 -- 172.21.15.143:0/131738740 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef002ca70 con 0x7fdef8160f70 2024-01-16T15:01:56.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.940+0000 7fdef5400640 1 -- 172.21.15.143:0/131738740 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef002cbd0 con 0x7fdef8160f70 2024-01-16T15:01:56.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.940+0000 7fdef5400640 1 -- 172.21.15.143:0/131738740 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef00343f0 con 0x7fdef8160f70 2024-01-16T15:01:56.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.941+0000 7fdef7200640 1 -- 172.21.15.143:0/131738740 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdef8160f70 msgr2=0x7fdef81613d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:56.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.941+0000 7fdef7200640 1 --2- 172.21.15.143:0/131738740 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdef8160f70 0x7fdef81613d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdef80ccb30 tx=0x7fdef002bed0 comp rx=0 tx=0).stop 2024-01-16T15:01:56.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.941+0000 7fdef7200640 1 -- 172.21.15.143:0/131738740 shutdown_connections 2024-01-16T15:01:56.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.941+0000 7fdef7200640 1 --2- 172.21.15.143:0/131738740 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdef8160f70 0x7fdef81613d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:56.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.941+0000 7fdef7200640 1 --2- 172.21.15.143:0/131738740 >> v2:172.21.15.204:3300/0 conn(0x7fdef80b0b40 0x7fdef80b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:56.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.941+0000 7fdef7200640 1 --2- 172.21.15.143:0/131738740 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdef815d310 0x7fdef80ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:56.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.941+0000 7fdef7200640 1 -- 172.21.15.143:0/131738740 >> 172.21.15.143:0/131738740 conn(0x7fdef80cbb40 msgr2=0x7fdef80cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:56.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.941+0000 7fdef7200640 1 -- 172.21.15.143:0/131738740 shutdown_connections 2024-01-16T15:01:56.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.941+0000 7fdef7200640 1 -- 172.21.15.143:0/131738740 wait complete. 2024-01-16T15:01:56.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.943+0000 7fdef7200640 1 Processor -- start 2024-01-16T15:01:56.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.943+0000 7fdef7200640 1 -- start start 2024-01-16T15:01:56.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.944+0000 7fdef7200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdef80b0b40 0x7fdef8149400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:56.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.944+0000 7fdef7200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdef815d310 0x7fdef815b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:56.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.944+0000 7fdef7200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdef8160f70 0x7fdef80c3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:56.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.944+0000 7fdef7200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef815bfe0 con 0x7fdef80b0b40 2024-01-16T15:01:56.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.944+0000 7fdef7200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef815c2c0 con 0x7fdef815d310 2024-01-16T15:01:56.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.944+0000 7fdef7200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef815c5a0 con 0x7fdef8160f70 2024-01-16T15:01:56.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.944+0000 7fdef5e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdef80b0b40 0x7fdef8149400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:56.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.944+0000 7fdef5e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdef80b0b40 0x7fdef8149400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39420/0 (socket says 172.21.15.143:39420) 2024-01-16T15:01:56.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.944+0000 7fdef5e00640 1 -- 172.21.15.143:0/1084035149 learned_addr learned my addr 172.21.15.143:0/1084035149 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:01:56.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.945+0000 7fdef5e00640 1 -- 172.21.15.143:0/1084035149 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdef8160f70 msgr2=0x7fdef80c3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:56.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.945+0000 7fdef5e00640 1 --2- 172.21.15.143:0/1084035149 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdef8160f70 0x7fdef80c3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:56.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.945+0000 7fdeed200640 1 --2- 172.21.15.143:0/1084035149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdef815d310 0x7fdef815b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:56.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.945+0000 7fdef5e00640 1 -- 172.21.15.143:0/1084035149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdef815d310 msgr2=0x7fdef815b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:56.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.945+0000 7fdef5e00640 1 --2- 172.21.15.143:0/1084035149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdef815d310 0x7fdef815b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:56.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.945+0000 7fdef5e00640 1 -- 172.21.15.143:0/1084035149 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef0009b30 con 0x7fdef80b0b40 2024-01-16T15:01:56.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.945+0000 7fdef5e00640 1 --2- 172.21.15.143:0/1084035149 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdef80b0b40 0x7fdef8149400 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fdee800d5f0 tx=0x7fdee800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:56.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.945+0000 7fdeeea00640 1 -- 172.21.15.143:0/1084035149 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee80125f0 con 0x7fdef80b0b40 2024-01-16T15:01:56.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.945+0000 7fdeeea00640 1 -- 172.21.15.143:0/1084035149 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee8012750 con 0x7fdef80b0b40 2024-01-16T15:01:56.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.945+0000 7fdef7200640 1 -- 172.21.15.143:0/1084035149 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdef815c880 con 0x7fdef80b0b40 2024-01-16T15:01:56.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.945+0000 7fdeeea00640 1 -- 172.21.15.143:0/1084035149 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee8012b00 con 0x7fdef80b0b40 2024-01-16T15:01:56.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.945+0000 7fdef7200640 1 -- 172.21.15.143:0/1084035149 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef80c2e80 con 0x7fdef80b0b40 2024-01-16T15:01:56.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.947+0000 7fdef7200640 1 -- 172.21.15.143:0/1084035149 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdef804e840 con 0x7fdef80b0b40 2024-01-16T15:01:56.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.947+0000 7fdeeea00640 1 -- 172.21.15.143:0/1084035149 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 6) v1 ==== 89590+0+0 (secure 0 0 0) 0x7fdee8012c60 con 0x7fdef80b0b40 2024-01-16T15:01:56.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.948+0000 7fdeeea00640 1 --2- 172.21.15.143:0/1084035149 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fded006b350 0x7fded006d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:01:56.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.948+0000 7fdeeea00640 1 -- 172.21.15.143:0/1084035149 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdee808a1e0 con 0x7fdef80b0b40 2024-01-16T15:01:56.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.949+0000 7fdeed200640 1 --2- 172.21.15.143:0/1084035149 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fded006b350 0x7fded006d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:01:56.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.949+0000 7fdeed200640 1 --2- 172.21.15.143:0/1084035149 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fded006b350 0x7fded006d810 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdee0006fd0 tx=0x7fdee0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:01:56.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:56.953+0000 7fdeeea00640 1 -- 172.21.15.143:0/1084035149 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdee8054f60 con 0x7fdef80b0b40 2024-01-16T15:01:57.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:57.171+0000 7fdef7200640 1 -- 172.21.15.143:0/1084035149 --> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7fdef80cee10 con 0x7fded006b350 2024-01-16T15:01:57.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:57.184+0000 7fdeeea00640 1 -- 172.21.15.143:0/1084035149 <== mgr.4113 v2:172.21.15.143:6832/3153492115 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7fdef80cee10 con 0x7fded006b350 2024-01-16T15:01:57.164 INFO:teuthology.orchestra.run.smithi143.stdout:/volumes/_nogroup/sv1/35696085-26f8-4228-885d-3e94e8d48712 2024-01-16T15:01:57.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:57.187+0000 7fdef7200640 1 -- 172.21.15.143:0/1084035149 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fded006b350 msgr2=0x7fded006d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:57.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:57.187+0000 7fdef7200640 1 --2- 172.21.15.143:0/1084035149 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fded006b350 0x7fded006d810 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdee0006fd0 tx=0x7fdee0008040 comp rx=0 tx=0).stop 2024-01-16T15:01:57.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:57.187+0000 7fdef7200640 1 -- 172.21.15.143:0/1084035149 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdef80b0b40 msgr2=0x7fdef8149400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:01:57.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:57.187+0000 7fdef7200640 1 --2- 172.21.15.143:0/1084035149 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdef80b0b40 0x7fdef8149400 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fdee800d5f0 tx=0x7fdee800dab0 comp rx=0 tx=0).stop 2024-01-16T15:01:57.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:57.187+0000 7fdef7200640 1 -- 172.21.15.143:0/1084035149 shutdown_connections 2024-01-16T15:01:57.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:57.187+0000 7fdef7200640 1 --2- 172.21.15.143:0/1084035149 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fded006b350 0x7fded006d810 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:57.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:57.187+0000 7fdef7200640 1 --2- 172.21.15.143:0/1084035149 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdef8160f70 0x7fdef80c3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:57.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:57.187+0000 7fdef7200640 1 --2- 172.21.15.143:0/1084035149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdef815d310 0x7fdef815b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:57.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:57.187+0000 7fdef7200640 1 --2- 172.21.15.143:0/1084035149 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdef80b0b40 0x7fdef8149400 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:01:57.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:57.187+0000 7fdef7200640 1 -- 172.21.15.143:0/1084035149 >> 172.21.15.143:0/1084035149 conn(0x7fdef80cbb40 msgr2=0x7fdef81e6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:01:57.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:57.188+0000 7fdef7200640 1 -- 172.21.15.143:0/1084035149 shutdown_connections 2024-01-16T15:01:57.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:01:57.188+0000 7fdef7200640 1 -- 172.21.15.143:0/1084035149 wait complete. 2024-01-16T15:01:57.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:01:57.186 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/35696085-26f8-4228-885d-3e94e8d48712) 2024-01-16T15:01:57.232 INFO:teuthology.orchestra.run.smithi143.stdout:# file: volumes/_nogroup/sv1/35696085-26f8-4228-885d-3e94e8d48712 2024-01-16T15:01:57.232 INFO:teuthology.orchestra.run.smithi143.stdout:ceph.dir.rbytes="3055190016" 2024-01-16T15:01:57.232 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:01:57.233 INFO:tasks.cephfs.test_volumes:r_size = 3055190016 2024-01-16T15:01:59.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:01:59.234 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/35696085-26f8-4228-885d-3e94e8d48712) 2024-01-16T15:01:59.292 INFO:teuthology.orchestra.run.smithi143.stdout:# file: volumes/_nogroup/sv1/35696085-26f8-4228-885d-3e94e8d48712 2024-01-16T15:01:59.293 INFO:teuthology.orchestra.run.smithi143.stdout:ceph.dir.rbytes="3055190016" 2024-01-16T15:01:59.293 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:01:59.294 INFO:tasks.cephfs.test_volumes:r_size = 3055190016 2024-01-16T15:02:01.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:02:01.295 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/35696085-26f8-4228-885d-3e94e8d48712) 2024-01-16T15:02:01.354 INFO:teuthology.orchestra.run.smithi143.stdout:# file: volumes/_nogroup/sv1/35696085-26f8-4228-885d-3e94e8d48712 2024-01-16T15:02:01.354 INFO:teuthology.orchestra.run.smithi143.stdout:ceph.dir.rbytes="3221225472" 2024-01-16T15:02:01.354 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:02:01.355 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 2024-01-16T15:02:01.355 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 2024-01-16T15:02:01.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.595+0000 7f72e6600640 1 Processor -- start 2024-01-16T15:02:01.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.595+0000 7f72e6600640 1 -- start start 2024-01-16T15:02:01.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.597+0000 7f72e6600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f72e00c59b0 0x7f72e0166230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:01.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.597+0000 7f72e6600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72e00c5dc0 con 0x7f72e015b440 2024-01-16T15:02:01.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.597+0000 7f72e6600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72e015eea0 con 0x7f72e00cc0f0 2024-01-16T15:02:01.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.597+0000 7f72e6600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72e015b820 con 0x7f72e00c59b0 2024-01-16T15:02:01.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.597+0000 7f72dfe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f72e00c59b0 0x7f72e0166230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:01.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.597+0000 7f72dfe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f72e00c59b0 0x7f72e0166230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39432/0 (socket says 172.21.15.143:39432) 2024-01-16T15:02:01.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.597+0000 7f72dfe00640 1 -- 172.21.15.143:0/3930073099 learned_addr learned my addr 172.21.15.143:0/3930073099 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:02:01.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.597+0000 7f72df400640 1 --1- 172.21.15.143:0/3930073099 >> v1:172.21.15.143:6789/0 conn(0x7f72e015b440 0x7f72e00cb9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51942/0 (socket says 172.21.15.143:51942) 2024-01-16T15:02:01.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.598+0000 7f72dfe00640 1 -- 172.21.15.143:0/3930073099 >> v1:172.21.15.204:6789/0 conn(0x7f72e00cc0f0 legacy=0x7f72e00c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:01.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.598+0000 7f72dfe00640 1 -- 172.21.15.143:0/3930073099 >> v1:172.21.15.143:6789/0 conn(0x7f72e015b440 legacy=0x7f72e00cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:01.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.598+0000 7f72dfe00640 1 -- 172.21.15.143:0/3930073099 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e006d600 con 0x7f72e00c59b0 2024-01-16T15:02:01.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.598+0000 7f72dea00640 1 -- 172.21.15.143:0/3930073099 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 389769426 0 0) 0x7f72e00c5dc0 con 0x7f72e015b440 2024-01-16T15:02:01.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.599+0000 7f72dfe00640 1 --2- 172.21.15.143:0/3930073099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72e00c59b0 0x7f72e0166230 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f72d8009e50 tx=0x7f72d802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16437f6a79b6acc6 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:01.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.599+0000 7f72dea00640 1 -- 172.21.15.143:0/3930073099 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d802ca80 con 0x7f72e00c59b0 2024-01-16T15:02:01.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.599+0000 7f72dea00640 1 -- 172.21.15.143:0/3930073099 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d802cbe0 con 0x7f72e00c59b0 2024-01-16T15:02:01.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.599+0000 7f72dea00640 1 -- 172.21.15.143:0/3930073099 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d80343f0 con 0x7f72e00c59b0 2024-01-16T15:02:01.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.600+0000 7f72e6600640 1 -- 172.21.15.143:0/3930073099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72e00c59b0 msgr2=0x7f72e0166230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:01.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.600+0000 7f72e6600640 1 --2- 172.21.15.143:0/3930073099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72e00c59b0 0x7f72e0166230 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f72d8009e50 tx=0x7f72d802bee0 comp rx=0 tx=0).stop 2024-01-16T15:02:01.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.600+0000 7f72e6600640 1 -- 172.21.15.143:0/3930073099 shutdown_connections 2024-01-16T15:02:01.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.600+0000 7f72e6600640 1 --2- 172.21.15.143:0/3930073099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72e00c59b0 0x7f72e0166230 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:01.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.600+0000 7f72e6600640 1 -- 172.21.15.143:0/3930073099 >> 172.21.15.143:0/3930073099 conn(0x7f72e00b0c50 msgr2=0x7f72e00b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:01.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.600+0000 7f72e6600640 1 -- 172.21.15.143:0/3930073099 shutdown_connections 2024-01-16T15:02:01.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.600+0000 7f72e6600640 1 -- 172.21.15.143:0/3930073099 wait complete. 2024-01-16T15:02:01.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.602+0000 7f72e6600640 1 Processor -- start 2024-01-16T15:02:01.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.602+0000 7f72e6600640 1 -- start start 2024-01-16T15:02:01.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.603+0000 7f72e6600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72e00c59b0 0x7f72e0154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:01.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.603+0000 7f72e6600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f72e00cc0f0 0x7f72e0070810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:01.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.603+0000 7f72e6600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72e015b440 0x7f72e0070d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:01.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.603+0000 7f72e6600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e0154ff0 con 0x7f72e00c59b0 2024-01-16T15:02:01.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.603+0000 7f72e6600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e00ccbd0 con 0x7f72e015b440 2024-01-16T15:02:01.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.603+0000 7f72e6600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72e00cd750 con 0x7f72e00cc0f0 2024-01-16T15:02:01.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.603+0000 7f72df400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72e00c59b0 0x7f72e0154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:01.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.603+0000 7f72df400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72e00c59b0 0x7f72e0154ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39438/0 (socket says 172.21.15.143:39438) 2024-01-16T15:02:01.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.603+0000 7f72df400640 1 -- 172.21.15.143:0/1116637242 learned_addr learned my addr 172.21.15.143:0/1116637242 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:02:01.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.603+0000 7f72df400640 1 -- 172.21.15.143:0/1116637242 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f72e00cc0f0 msgr2=0x7f72e0070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:01.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.603+0000 7f72df400640 1 --2- 172.21.15.143:0/1116637242 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f72e00cc0f0 0x7f72e0070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:01.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.603+0000 7f72df400640 1 -- 172.21.15.143:0/1116637242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72e015b440 msgr2=0x7f72e0070d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:01.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.603+0000 7f72df400640 1 --2- 172.21.15.143:0/1116637242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72e015b440 0x7f72e0070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:01.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.603+0000 7f72df400640 1 -- 172.21.15.143:0/1116637242 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d8009b30 con 0x7f72e00c59b0 2024-01-16T15:02:01.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.604+0000 7f72df400640 1 --2- 172.21.15.143:0/1116637242 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72e00c59b0 0x7f72e0154ab0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f72d000c460 tx=0x7f72d000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:01.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.604+0000 7f72d7e00640 1 -- 172.21.15.143:0/1116637242 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d00074e0 con 0x7f72e00c59b0 2024-01-16T15:02:01.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.604+0000 7f72d7e00640 1 -- 172.21.15.143:0/1116637242 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d0007660 con 0x7f72e00c59b0 2024-01-16T15:02:01.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.604+0000 7f72d7e00640 1 -- 172.21.15.143:0/1116637242 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d0004a20 con 0x7f72e00c59b0 2024-01-16T15:02:01.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.604+0000 7f72e6600640 1 -- 172.21.15.143:0/1116637242 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72e00cce20 con 0x7f72e00c59b0 2024-01-16T15:02:01.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.604+0000 7f72e6600640 1 -- 172.21.15.143:0/1116637242 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72e00cd3e0 con 0x7f72e00c59b0 2024-01-16T15:02:01.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.605+0000 7f72e6600640 1 -- 172.21.15.143:0/1116637242 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72e004e8b0 con 0x7f72e00c59b0 2024-01-16T15:02:01.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.606+0000 7f72d7e00640 1 -- 172.21.15.143:0/1116637242 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 6) v1 ==== 89590+0+0 (secure 0 0 0) 0x7f72d0004b80 con 0x7f72e00c59b0 2024-01-16T15:02:01.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.607+0000 7f72d7e00640 1 --2- 172.21.15.143:0/1116637242 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f72b806b350 0x7f72b806d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:01.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.607+0000 7f72d7e00640 1 -- 172.21.15.143:0/1116637242 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f72d0088f00 con 0x7f72e00c59b0 2024-01-16T15:02:01.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.607+0000 7f72d6800640 1 --2- 172.21.15.143:0/1116637242 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f72b806b350 0x7f72b806d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:01.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.608+0000 7f72d6800640 1 --2- 172.21.15.143:0/1116637242 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f72b806b350 0x7f72b806d810 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f72e00c9390 tx=0x7f72c80079b0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:01.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.612+0000 7f72d7e00640 1 -- 172.21.15.143:0/1116637242 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f72d0053c80 con 0x7f72e00c59b0 2024-01-16T15:02:01.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.839+0000 7f72e6600640 1 -- 172.21.15.143:0/1116637242 --> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f72e015f3d0 con 0x7f72b806b350 2024-01-16T15:02:01.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.984+0000 7f72d7e00640 1 -- 172.21.15.143:0/1116637242 <== mgr.4113 v2:172.21.15.143:6832/3153492115 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f72c8003950 con 0x7f72b806b350 2024-01-16T15:02:01.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.987+0000 7f72e6600640 1 -- 172.21.15.143:0/1116637242 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f72b806b350 msgr2=0x7f72b806d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:01.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.987+0000 7f72e6600640 1 --2- 172.21.15.143:0/1116637242 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f72b806b350 0x7f72b806d810 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f72e00c9390 tx=0x7f72c80079b0 comp rx=0 tx=0).stop 2024-01-16T15:02:01.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.987+0000 7f72e6600640 1 -- 172.21.15.143:0/1116637242 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72e00c59b0 msgr2=0x7f72e0154ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:01.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.987+0000 7f72e6600640 1 --2- 172.21.15.143:0/1116637242 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72e00c59b0 0x7f72e0154ab0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f72d000c460 tx=0x7f72d000c920 comp rx=0 tx=0).stop 2024-01-16T15:02:01.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.987+0000 7f72e6600640 1 -- 172.21.15.143:0/1116637242 shutdown_connections 2024-01-16T15:02:01.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.987+0000 7f72e6600640 1 --2- 172.21.15.143:0/1116637242 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f72b806b350 0x7f72b806d810 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:01.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.987+0000 7f72e6600640 1 --2- 172.21.15.143:0/1116637242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72e015b440 0x7f72e0070d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:01.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.987+0000 7f72e6600640 1 --2- 172.21.15.143:0/1116637242 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f72e00cc0f0 0x7f72e0070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:01.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.987+0000 7f72e6600640 1 --2- 172.21.15.143:0/1116637242 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72e00c59b0 0x7f72e0154ab0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:01.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.987+0000 7f72e6600640 1 -- 172.21.15.143:0/1116637242 >> 172.21.15.143:0/1116637242 conn(0x7f72e00b0c50 msgr2=0x7f72e00cb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:01.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.987+0000 7f72e6600640 1 -- 172.21.15.143:0/1116637242 shutdown_connections 2024-01-16T15:02:01.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:01.987+0000 7f72e6600640 1 -- 172.21.15.143:0/1116637242 wait complete. 2024-01-16T15:02:01.972 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T15:02:01.992+0000 7f2776e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T15:02:01.972 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T15:02:01.992+0000 7f2776e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T15:02:01.972 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T15:02:01.992+0000 7f2776e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T15:02:01.972 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T15:02:01.992+0000 7f2776e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T15:02:01.972 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T15:02:01.992+0000 7f2776e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T15:02:01.990 INFO:tasks.cephfs.test_volumes:expected clone state = ('in-progress',) 2024-01-16T15:02:01.991 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-16T15:02:01.992 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T15:02:02.013+0000 7f2776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T15:02:01.992 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T15:02:02.013+0000 7f2776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T15:02:01.992 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T15:02:02.013+0000 7f2776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T15:02:01.992 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T15:02:02.013+0000 7f2776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T15:02:01.992 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T15:02:02.013+0000 7f2776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T15:02:02.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.204+0000 7f0bbba00640 1 Processor -- start 2024-01-16T15:02:02.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.204+0000 7f0bbba00640 1 -- start start 2024-01-16T15:02:02.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.205+0000 7f0bbba00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0bb415b440 0x7f0bb40cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:02.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.206+0000 7f0bbba00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0bb40cbf20 0x7f0bb40c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:02.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.206+0000 7f0bbba00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bb415eea0 con 0x7f0bb40c57e0 2024-01-16T15:02:02.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.206+0000 7f0bbba00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bb40cc3d0 con 0x7f0bb415b440 2024-01-16T15:02:02.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.206+0000 7f0bbba00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bb415b820 con 0x7f0bb40cbf20 2024-01-16T15:02:02.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.206+0000 7f0bb9000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0bb415b440 0x7f0bb40cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:02.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.206+0000 7f0bb9000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0bb415b440 0x7f0bb40cb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39472/0 (socket says 172.21.15.143:39472) 2024-01-16T15:02:02.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.206+0000 7f0bb9000640 1 -- 172.21.15.143:0/2966413775 learned_addr learned my addr 172.21.15.143:0/2966413775 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:02:02.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.206+0000 7f0bb3e00640 1 --2- 172.21.15.143:0/2966413775 >> v2:172.21.15.204:3300/0 conn(0x7f0bb40cbf20 0x7f0bb40c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:02.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.206+0000 7f0bb9000640 1 -- 172.21.15.143:0/2966413775 >> v2:172.21.15.204:3300/0 conn(0x7f0bb40cbf20 msgr2=0x7f0bb40c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:02.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.206+0000 7f0bb9000640 1 --2- 172.21.15.143:0/2966413775 >> v2:172.21.15.204:3300/0 conn(0x7f0bb40cbf20 0x7f0bb40c52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:02.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.206+0000 7f0bb9000640 1 -- 172.21.15.143:0/2966413775 >> v1:172.21.15.204:6789/0 conn(0x7f0bb40c57e0 legacy=0x7f0bb40c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:02.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.207+0000 7f0bb9000640 1 -- 172.21.15.143:0/2966413775 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb406d600 con 0x7f0bb415b440 2024-01-16T15:02:02.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.208+0000 7f0bb9000640 1 --2- 172.21.15.143:0/2966413775 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0bb415b440 0x7f0bb40cb9e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0bb406e890 tx=0x7f0ba402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48ae0b70922b312b server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:02.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.208+0000 7f0bb3400640 1 -- 172.21.15.143:0/2966413775 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba40317e0 con 0x7f0bb415b440 2024-01-16T15:02:02.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.208+0000 7f0bb3400640 1 -- 172.21.15.143:0/2966413775 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba4031940 con 0x7f0bb415b440 2024-01-16T15:02:02.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.208+0000 7f0bb3400640 1 -- 172.21.15.143:0/2966413775 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba4031cd0 con 0x7f0bb415b440 2024-01-16T15:02:02.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.209+0000 7f0bbba00640 1 -- 172.21.15.143:0/2966413775 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0bb415b440 msgr2=0x7f0bb40cb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:02.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.209+0000 7f0bbba00640 1 --2- 172.21.15.143:0/2966413775 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0bb415b440 0x7f0bb40cb9e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0bb406e890 tx=0x7f0ba402b880 comp rx=0 tx=0).stop 2024-01-16T15:02:02.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.209+0000 7f0bbba00640 1 -- 172.21.15.143:0/2966413775 shutdown_connections 2024-01-16T15:02:02.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.209+0000 7f0bbba00640 1 --2- 172.21.15.143:0/2966413775 >> v2:172.21.15.204:3300/0 conn(0x7f0bb40cbf20 0x7f0bb40c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:02.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.209+0000 7f0bbba00640 1 --2- 172.21.15.143:0/2966413775 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0bb415b440 0x7f0bb40cb9e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:02.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.209+0000 7f0bbba00640 1 -- 172.21.15.143:0/2966413775 >> 172.21.15.143:0/2966413775 conn(0x7f0bb40b0c50 msgr2=0x7f0bb40b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:02.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.209+0000 7f0bbba00640 1 -- 172.21.15.143:0/2966413775 shutdown_connections 2024-01-16T15:02:02.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.209+0000 7f0bbba00640 1 -- 172.21.15.143:0/2966413775 wait complete. 2024-01-16T15:02:02.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.211+0000 7f0bbba00640 1 Processor -- start 2024-01-16T15:02:02.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.211+0000 7f0bbba00640 1 -- start start 2024-01-16T15:02:02.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.212+0000 7f0bbba00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bb40c57e0 0x7f0bb41f4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:02.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.212+0000 7f0bbba00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0bb40cbf20 0x7f0bb40b3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:02.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.212+0000 7f0bbba00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0bb415b440 0x7f0bb40b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:02.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.212+0000 7f0bbba00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb40c6b30 con 0x7f0bb40cbf20 2024-01-16T15:02:02.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.212+0000 7f0bbba00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb40c8e00 con 0x7f0bb40c57e0 2024-01-16T15:02:02.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.212+0000 7f0bbba00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb40b4520 con 0x7f0bb415b440 2024-01-16T15:02:02.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.212+0000 7f0bb3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0bb40cbf20 0x7f0bb40b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:02.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.212+0000 7f0bb3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0bb40cbf20 0x7f0bb40b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39484/0 (socket says 172.21.15.143:39484) 2024-01-16T15:02:02.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.212+0000 7f0bb3e00640 1 -- 172.21.15.143:0/3999945255 learned_addr learned my addr 172.21.15.143:0/3999945255 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:02:02.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.213+0000 7f0bb9a00640 1 --2- 172.21.15.143:0/3999945255 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0bb415b440 0x7f0bb40b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:02.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.213+0000 7f0bb3e00640 1 -- 172.21.15.143:0/3999945255 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0bb415b440 msgr2=0x7f0bb40b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:02.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.213+0000 7f0bb9000640 1 --2- 172.21.15.143:0/3999945255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bb40c57e0 0x7f0bb41f4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:02.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.213+0000 7f0bb3e00640 1 --2- 172.21.15.143:0/3999945255 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0bb415b440 0x7f0bb40b3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:02.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.213+0000 7f0bb3e00640 1 -- 172.21.15.143:0/3999945255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bb40c57e0 msgr2=0x7f0bb41f4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:02.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.213+0000 7f0bb3e00640 1 --2- 172.21.15.143:0/3999945255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bb40c57e0 0x7f0bb41f4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:02.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.213+0000 7f0bb3e00640 1 -- 172.21.15.143:0/3999945255 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba40094d0 con 0x7f0bb40cbf20 2024-01-16T15:02:02.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.213+0000 7f0bb3e00640 1 --2- 172.21.15.143:0/3999945255 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0bb40cbf20 0x7f0bb40b3940 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c00d830 tx=0x7f0b9c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:02.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.213+0000 7f0bb0c00640 1 -- 172.21.15.143:0/3999945255 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c00b6c0 con 0x7f0bb40cbf20 2024-01-16T15:02:02.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.213+0000 7f0bb0c00640 1 -- 172.21.15.143:0/3999945255 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b9c00b820 con 0x7f0bb40cbf20 2024-01-16T15:02:02.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.213+0000 7f0bb0c00640 1 -- 172.21.15.143:0/3999945255 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c00bbb0 con 0x7f0bb40cbf20 2024-01-16T15:02:02.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.213+0000 7f0bbba00640 1 -- 172.21.15.143:0/3999945255 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bb40c6200 con 0x7f0bb40cbf20 2024-01-16T15:02:02.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.213+0000 7f0bbba00640 1 -- 172.21.15.143:0/3999945255 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bb40c67c0 con 0x7f0bb40cbf20 2024-01-16T15:02:02.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.215+0000 7f0bbba00640 1 -- 172.21.15.143:0/3999945255 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bb40c6ce0 con 0x7f0bb40cbf20 2024-01-16T15:02:02.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.215+0000 7f0bb0c00640 1 -- 172.21.15.143:0/3999945255 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 6) v1 ==== 89590+0+0 (secure 0 0 0) 0x7f0b9c01e460 con 0x7f0bb40cbf20 2024-01-16T15:02:02.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.216+0000 7f0bb0c00640 1 --2- 172.21.15.143:0/3999945255 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0b8c06b350 0x7f0b8c06d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:02.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.216+0000 7f0bb0c00640 1 -- 172.21.15.143:0/3999945255 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0b9c08a3e0 con 0x7f0bb40cbf20 2024-01-16T15:02:02.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.217+0000 7f0bb9000640 1 --2- 172.21.15.143:0/3999945255 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0b8c06b350 0x7f0b8c06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:02.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.217+0000 7f0bb9000640 1 --2- 172.21.15.143:0/3999945255 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0b8c06b350 0x7f0b8c06d810 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4036000 tx=0x7f0ba403d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:02.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.221+0000 7f0bb0c00640 1 -- 172.21.15.143:0/3999945255 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0bb40c6ce0 con 0x7f0bb40cbf20 2024-01-16T15:02:02.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:02.448+0000 7f0bbba00640 1 -- 172.21.15.143:0/3999945255 --> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f0bb415f3d0 con 0x7f0b8c06b350 2024-01-16T15:02:04.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:04.619+0000 7f0bb0c00640 1 -- 172.21.15.143:0/3999945255 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7f0b9c0555c0 con 0x7f0bb40cbf20 2024-01-16T15:02:05.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.060+0000 7f0bb0c00640 1 -- 172.21.15.143:0/3999945255 <== mgr.4113 v2:172.21.15.143:6832/3153492115 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7f0ba4031b30 con 0x7f0b8c06b350 2024-01-16T15:02:05.041 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-16T15:02:05.041 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-16T15:02:05.041 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-16T15:02:05.041 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-16T15:02:05.041 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-16T15:02:05.041 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-16T15:02:05.041 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-16T15:02:05.042 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:02:05.042 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:02:05.042 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-16T15:02:05.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.064+0000 7f0bbba00640 1 -- 172.21.15.143:0/3999945255 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0b8c06b350 msgr2=0x7f0b8c06d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:05.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.064+0000 7f0bbba00640 1 --2- 172.21.15.143:0/3999945255 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0b8c06b350 0x7f0b8c06d810 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4036000 tx=0x7f0ba403d040 comp rx=0 tx=0).stop 2024-01-16T15:02:05.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.064+0000 7f0bbba00640 1 -- 172.21.15.143:0/3999945255 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0bb40cbf20 msgr2=0x7f0bb40b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:05.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.064+0000 7f0bbba00640 1 --2- 172.21.15.143:0/3999945255 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0bb40cbf20 0x7f0bb40b3940 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c00d830 tx=0x7f0b9c00dcf0 comp rx=0 tx=0).stop 2024-01-16T15:02:05.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.065+0000 7f0bbba00640 1 -- 172.21.15.143:0/3999945255 shutdown_connections 2024-01-16T15:02:05.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.065+0000 7f0bbba00640 1 --2- 172.21.15.143:0/3999945255 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f0b8c06b350 0x7f0b8c06d810 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:05.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.065+0000 7f0bbba00640 1 --2- 172.21.15.143:0/3999945255 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0bb415b440 0x7f0bb40b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:05.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.065+0000 7f0bbba00640 1 --2- 172.21.15.143:0/3999945255 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0bb40cbf20 0x7f0bb40b3940 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:05.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.065+0000 7f0bbba00640 1 --2- 172.21.15.143:0/3999945255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bb40c57e0 0x7f0bb41f4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:05.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.065+0000 7f0bbba00640 1 -- 172.21.15.143:0/3999945255 >> 172.21.15.143:0/3999945255 conn(0x7f0bb40b0c50 msgr2=0x7f0bb406cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:05.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.065+0000 7f0bbba00640 1 -- 172.21.15.143:0/3999945255 shutdown_connections 2024-01-16T15:02:05.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.065+0000 7f0bbba00640 1 -- 172.21.15.143:0/3999945255 wait complete. 2024-01-16T15:02:05.070 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-01-16T15:02:05.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.289+0000 7fb989e00640 1 Processor -- start 2024-01-16T15:02:05.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.289+0000 7fb989e00640 1 -- start start 2024-01-16T15:02:05.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.290+0000 7fb989e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb9840b2250 0x7fb98415df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:05.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.290+0000 7fb989e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb98415ab80 con 0x7fb9840b2aa0 2024-01-16T15:02:05.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.290+0000 7fb989e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb98415e680 con 0x7fb9840b2630 2024-01-16T15:02:05.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.290+0000 7fb989e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb98415a990 con 0x7fb9840b2250 2024-01-16T15:02:05.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.290+0000 7fb983400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb9840b2aa0 0x7fb98415b080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:38236/0 (socket says 172.21.15.143:38236) 2024-01-16T15:02:05.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.291+0000 7fb983400640 1 -- 172.21.15.143:0/1075586240 learned_addr learned my addr 172.21.15.143:0/1075586240 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:02:05.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.291+0000 7fb982a00640 1 --2- 172.21.15.143:0/1075586240 >> v2:172.21.15.204:3300/0 conn(0x7fb9840b2250 0x7fb98415df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:05.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.291+0000 7fb981600640 1 -- 172.21.15.143:0/1075586240 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1446239915 0 0) 0x7fb98415ab80 con 0x7fb9840b2aa0 2024-01-16T15:02:05.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.291+0000 7fb981600640 1 -- 172.21.15.143:0/1075586240 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb964003520 con 0x7fb9840b2aa0 2024-01-16T15:02:05.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.291+0000 7fb981600640 1 -- 172.21.15.143:0/1075586240 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2631607571 0 0) 0x7fb964003520 con 0x7fb9840b2aa0 2024-01-16T15:02:05.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.291+0000 7fb981600640 1 -- 172.21.15.143:0/1075586240 >> v2:172.21.15.204:3300/0 conn(0x7fb9840b2250 msgr2=0x7fb98415df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:05.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.291+0000 7fb981600640 1 --2- 172.21.15.143:0/1075586240 >> v2:172.21.15.204:3300/0 conn(0x7fb9840b2250 0x7fb98415df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:05.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.291+0000 7fb981600640 1 -- 172.21.15.143:0/1075586240 >> v1:172.21.15.204:6789/0 conn(0x7fb9840b2630 legacy=0x7fb9840ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:05.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.291+0000 7fb981600640 1 -- 172.21.15.143:0/1075586240 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb98415e960 con 0x7fb9840b2aa0 2024-01-16T15:02:05.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.291+0000 7fb981600640 1 -- 172.21.15.143:0/1075586240 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7fb97c0019f0 con 0x7fb9840b2aa0 2024-01-16T15:02:05.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.291+0000 7fb981600640 1 -- 172.21.15.143:0/1075586240 >> v1:172.21.15.143:6789/0 conn(0x7fb9840b2aa0 legacy=0x7fb98415b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:05.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.291+0000 7fb981600640 1 --2- 172.21.15.143:0/1075586240 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9640039c0 0x7fb964003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:05.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.291+0000 7fb981600640 1 --2- 172.21.15.143:0/1075586240 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9640047c0 0x7fb964004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:05.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.292+0000 7fb981600640 1 --2- 172.21.15.143:0/1075586240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb964005180 0x7fb96400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:05.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.292+0000 7fb981600640 1 -- 172.21.15.143:0/1075586240 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96400c050 con 0x7fb9640039c0 2024-01-16T15:02:05.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.292+0000 7fb981600640 1 -- 172.21.15.143:0/1075586240 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96400c1c0 con 0x7fb964005180 2024-01-16T15:02:05.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.292+0000 7fb981600640 1 -- 172.21.15.143:0/1075586240 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb96400c380 con 0x7fb9640047c0 2024-01-16T15:02:05.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.292+0000 7fb982a00640 1 --2- 172.21.15.143:0/1075586240 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9640039c0 0x7fb964003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:05.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.292+0000 7fb982a00640 1 -- 172.21.15.143:0/1075586240 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9640047c0 msgr2=0x7fb964004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:05.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.292+0000 7fb982a00640 1 --2- 172.21.15.143:0/1075586240 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9640047c0 0x7fb964004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:05.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.292+0000 7fb982a00640 1 -- 172.21.15.143:0/1075586240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb964005180 msgr2=0x7fb96400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:05.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.292+0000 7fb982a00640 1 --2- 172.21.15.143:0/1075586240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb964005180 0x7fb96400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:05.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.292+0000 7fb982a00640 1 -- 172.21.15.143:0/1075586240 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb964003520 con 0x7fb9640039c0 2024-01-16T15:02:05.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.292+0000 7fb983400640 1 -- 172.21.15.143:0/1075586240 reap_dead start 2024-01-16T15:02:05.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.293+0000 7fb982a00640 1 --2- 172.21.15.143:0/1075586240 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9640039c0 0x7fb964003de0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb974009ac0 tx=0x7fb9640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d1ee12e982938a5 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:05.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.293+0000 7fb981600640 1 -- 172.21.15.143:0/1075586240 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97402c6b0 con 0x7fb9640039c0 2024-01-16T15:02:05.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.293+0000 7fb981600640 1 -- 172.21.15.143:0/1075586240 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb97402c810 con 0x7fb9640039c0 2024-01-16T15:02:05.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.293+0000 7fb981600640 1 -- 172.21.15.143:0/1075586240 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97402cba0 con 0x7fb9640039c0 2024-01-16T15:02:05.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.294+0000 7fb989e00640 1 -- 172.21.15.143:0/1075586240 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9640039c0 msgr2=0x7fb964003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:05.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.294+0000 7fb989e00640 1 --2- 172.21.15.143:0/1075586240 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9640039c0 0x7fb964003de0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb974009ac0 tx=0x7fb9640057f0 comp rx=0 tx=0).stop 2024-01-16T15:02:05.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.294+0000 7fb989e00640 1 -- 172.21.15.143:0/1075586240 shutdown_connections 2024-01-16T15:02:05.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.294+0000 7fb989e00640 1 --2- 172.21.15.143:0/1075586240 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9640039c0 0x7fb964003de0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:05.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.294+0000 7fb989e00640 1 -- 172.21.15.143:0/1075586240 >> 172.21.15.143:0/1075586240 conn(0x7fb98406a920 msgr2=0x7fb98406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:05.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.294+0000 7fb989e00640 1 -- 172.21.15.143:0/1075586240 shutdown_connections 2024-01-16T15:02:05.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.294+0000 7fb989e00640 1 -- 172.21.15.143:0/1075586240 wait complete. 2024-01-16T15:02:05.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.296+0000 7fb989e00640 1 Processor -- start 2024-01-16T15:02:05.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.296+0000 7fb989e00640 1 -- start start 2024-01-16T15:02:05.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.297+0000 7fb989e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9640039c0 0x7fb98415bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:05.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.297+0000 7fb989e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb98415c510 0x7fb9840717d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:05.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.297+0000 7fb989e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb984071d10 0x7fb9840721a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:05.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.297+0000 7fb989e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb984146620 con 0x7fb984071d10 2024-01-16T15:02:05.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.297+0000 7fb989e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98415c970 con 0x7fb98415c510 2024-01-16T15:02:05.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.297+0000 7fb989e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9841f3410 con 0x7fb9640039c0 2024-01-16T15:02:05.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.297+0000 7fb983400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb984071d10 0x7fb9840721a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:05.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.297+0000 7fb983400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb984071d10 0x7fb9840721a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60828/0 (socket says 172.21.15.143:60828) 2024-01-16T15:02:05.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.297+0000 7fb983400640 1 -- 172.21.15.143:0/192553059 learned_addr learned my addr 172.21.15.143:0/192553059 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:02:05.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.298+0000 7fb982a00640 1 --2- 172.21.15.143:0/192553059 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9640039c0 0x7fb98415bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:05.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.298+0000 7fb982000640 1 --2- 172.21.15.143:0/192553059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb98415c510 0x7fb9840717d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:05.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.298+0000 7fb983400640 1 -- 172.21.15.143:0/192553059 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9640039c0 msgr2=0x7fb98415bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:05.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.298+0000 7fb983400640 1 --2- 172.21.15.143:0/192553059 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9640039c0 0x7fb98415bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:05.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.298+0000 7fb983400640 1 -- 172.21.15.143:0/192553059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb98415c510 msgr2=0x7fb9840717d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:05.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.298+0000 7fb983400640 1 --2- 172.21.15.143:0/192553059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb98415c510 0x7fb9840717d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:05.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.298+0000 7fb983400640 1 -- 172.21.15.143:0/192553059 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9840705c0 con 0x7fb984071d10 2024-01-16T15:02:05.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.298+0000 7fb983400640 1 --2- 172.21.15.143:0/192553059 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb984071d10 0x7fb9840721a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb9840d0040 tx=0x7fb97c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:05.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.298+0000 7fb97b400640 1 -- 172.21.15.143:0/192553059 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97c004d70 con 0x7fb984071d10 2024-01-16T15:02:05.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.298+0000 7fb97b400640 1 -- 172.21.15.143:0/192553059 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb97c00a640 con 0x7fb984071d10 2024-01-16T15:02:05.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.298+0000 7fb989e00640 1 -- 172.21.15.143:0/192553059 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9840c8500 con 0x7fb984071d10 2024-01-16T15:02:05.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.298+0000 7fb97b400640 1 -- 172.21.15.143:0/192553059 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97c00a9f0 con 0x7fb984071d10 2024-01-16T15:02:05.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.299+0000 7fb989e00640 1 -- 172.21.15.143:0/192553059 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9840c8a80 con 0x7fb984071d10 2024-01-16T15:02:05.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.300+0000 7fb989e00640 1 -- 172.21.15.143:0/192553059 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb98406eed0 con 0x7fb984071d10 2024-01-16T15:02:05.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.301+0000 7fb97b400640 1 -- 172.21.15.143:0/192553059 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7fb97c00ab50 con 0x7fb984071d10 2024-01-16T15:02:05.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.301+0000 7fb97b400640 1 --2- 172.21.15.143:0/192553059 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb95c06b4a0 0x7fb95c06d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:05.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.302+0000 7fb97b400640 1 -- 172.21.15.143:0/192553059 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb97c089680 con 0x7fb984071d10 2024-01-16T15:02:05.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.302+0000 7fb982a00640 1 --2- 172.21.15.143:0/192553059 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb95c06b4a0 0x7fb95c06d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:05.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.302+0000 7fb982a00640 1 --2- 172.21.15.143:0/192553059 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb95c06b4a0 0x7fb95c06d960 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb974036000 tx=0x7fb974030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:05.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.306+0000 7fb97b400640 1 -- 172.21.15.143:0/192553059 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb97c0541d0 con 0x7fb984071d10 2024-01-16T15:02:05.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.571+0000 7fb989e00640 1 -- 172.21.15.143:0/192553059 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb984067470 con 0x7fb984071d10 2024-01-16T15:02:05.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.572+0000 7fb97b400640 1 -- 172.21.15.143:0/192553059 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2279 (secure 0 0 0) 0x7fb97c0577f0 con 0x7fb984071d10 2024-01-16T15:02:05.552 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:02:05.552 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-16T15:02:05.553 INFO:teuthology.orchestra.run.smithi143.stdout: "fsid": "c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0", 2024-01-16T15:02:05.553 INFO:teuthology.orchestra.run.smithi143.stdout: "health": { 2024-01-16T15:02:05.553 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "HEALTH_OK", 2024-01-16T15:02:05.553 INFO:teuthology.orchestra.run.smithi143.stdout: "checks": {}, 2024-01-16T15:02:05.553 INFO:teuthology.orchestra.run.smithi143.stdout: "mutes": [] 2024-01-16T15:02:05.553 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:02:05.553 INFO:teuthology.orchestra.run.smithi143.stdout: "election_epoch": 4, 2024-01-16T15:02:05.553 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum": [ 2024-01-16T15:02:05.553 INFO:teuthology.orchestra.run.smithi143.stdout: 0, 2024-01-16T15:02:05.553 INFO:teuthology.orchestra.run.smithi143.stdout: 1, 2024-01-16T15:02:05.553 INFO:teuthology.orchestra.run.smithi143.stdout: 2 2024-01-16T15:02:05.553 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-16T15:02:05.553 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_names": [ 2024-01-16T15:02:05.553 INFO:teuthology.orchestra.run.smithi143.stdout: "a", 2024-01-16T15:02:05.553 INFO:teuthology.orchestra.run.smithi143.stdout: "b", 2024-01-16T15:02:05.554 INFO:teuthology.orchestra.run.smithi143.stdout: "c" 2024-01-16T15:02:05.554 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-16T15:02:05.554 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_age": 147, 2024-01-16T15:02:05.554 INFO:teuthology.orchestra.run.smithi143.stdout: "monmap": { 2024-01-16T15:02:05.554 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 1, 2024-01-16T15:02:05.554 INFO:teuthology.orchestra.run.smithi143.stdout: "min_mon_release_name": "reef", 2024-01-16T15:02:05.554 INFO:teuthology.orchestra.run.smithi143.stdout: "num_mons": 3 2024-01-16T15:02:05.554 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:02:05.554 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap": { 2024-01-16T15:02:05.554 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 34, 2024-01-16T15:02:05.554 INFO:teuthology.orchestra.run.smithi143.stdout: "num_osds": 8, 2024-01-16T15:02:05.554 INFO:teuthology.orchestra.run.smithi143.stdout: "num_up_osds": 8, 2024-01-16T15:02:05.554 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_up_since": 1705417190, 2024-01-16T15:02:05.554 INFO:teuthology.orchestra.run.smithi143.stdout: "num_in_osds": 8, 2024-01-16T15:02:05.555 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_in_since": 1705417182, 2024-01-16T15:02:05.555 INFO:teuthology.orchestra.run.smithi143.stdout: "num_remapped_pgs": 0 2024-01-16T15:02:05.555 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:02:05.555 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap": { 2024-01-16T15:02:05.555 INFO:teuthology.orchestra.run.smithi143.stdout: "pgs_by_state": [ 2024-01-16T15:02:05.555 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-16T15:02:05.555 INFO:teuthology.orchestra.run.smithi143.stdout: "state_name": "active+clean", 2024-01-16T15:02:05.555 INFO:teuthology.orchestra.run.smithi143.stdout: "count": 97 2024-01-16T15:02:05.555 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:02:05.555 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-16T15:02:05.555 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pgs": 97, 2024-01-16T15:02:05.555 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pools": 3, 2024-01-16T15:02:05.555 INFO:teuthology.orchestra.run.smithi143.stdout: "num_objects": 793, 2024-01-16T15:02:05.555 INFO:teuthology.orchestra.run.smithi143.stdout: "data_bytes": 3222019561, 2024-01-16T15:02:05.555 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_used": 6698323968, 2024-01-16T15:02:05.555 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_avail": 766395789312, 2024-01-16T15:02:05.556 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_total": 773094113280, 2024-01-16T15:02:05.556 INFO:teuthology.orchestra.run.smithi143.stdout: "write_bytes_sec": 139770399, 2024-01-16T15:02:05.556 INFO:teuthology.orchestra.run.smithi143.stdout: "read_op_per_sec": 0, 2024-01-16T15:02:05.556 INFO:teuthology.orchestra.run.smithi143.stdout: "write_op_per_sec": 35 2024-01-16T15:02:05.556 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:02:05.556 INFO:teuthology.orchestra.run.smithi143.stdout: "fsmap": { 2024-01-16T15:02:05.556 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 18, 2024-01-16T15:02:05.556 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 2, 2024-01-16T15:02:05.556 INFO:teuthology.orchestra.run.smithi143.stdout: "up": 1, 2024-01-16T15:02:05.556 INFO:teuthology.orchestra.run.smithi143.stdout: "in": 1, 2024-01-16T15:02:05.556 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 1, 2024-01-16T15:02:05.556 INFO:teuthology.orchestra.run.smithi143.stdout: "by_rank": [ 2024-01-16T15:02:05.556 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-16T15:02:05.556 INFO:teuthology.orchestra.run.smithi143.stdout: "filesystem_id": 2, 2024-01-16T15:02:05.556 INFO:teuthology.orchestra.run.smithi143.stdout: "rank": 0, 2024-01-16T15:02:05.557 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "a", 2024-01-16T15:02:05.557 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "up:active", 2024-01-16T15:02:05.557 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 5094 2024-01-16T15:02:05.557 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:02:05.557 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-16T15:02:05.557 INFO:teuthology.orchestra.run.smithi143.stdout: "up:standby": 3 2024-01-16T15:02:05.557 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:02:05.557 INFO:teuthology.orchestra.run.smithi143.stdout: "mgrmap": { 2024-01-16T15:02:05.557 INFO:teuthology.orchestra.run.smithi143.stdout: "available": true, 2024-01-16T15:02:05.557 INFO:teuthology.orchestra.run.smithi143.stdout: "num_standbys": 1, 2024-01-16T15:02:05.557 INFO:teuthology.orchestra.run.smithi143.stdout: "modules": [ 2024-01-16T15:02:05.557 INFO:teuthology.orchestra.run.smithi143.stdout: "iostat", 2024-01-16T15:02:05.557 INFO:teuthology.orchestra.run.smithi143.stdout: "nfs", 2024-01-16T15:02:05.557 INFO:teuthology.orchestra.run.smithi143.stdout: "restful" 2024-01-16T15:02:05.557 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-16T15:02:05.557 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-16T15:02:05.557 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:02:05.558 INFO:teuthology.orchestra.run.smithi143.stdout: "servicemap": { 2024-01-16T15:02:05.558 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 4, 2024-01-16T15:02:05.558 INFO:teuthology.orchestra.run.smithi143.stdout: "modified": "2024-01-16T15:00:59.670524+0000", 2024-01-16T15:02:05.558 INFO:teuthology.orchestra.run.smithi143.stdout: "services": { 2024-01-16T15:02:05.558 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": { 2024-01-16T15:02:05.558 INFO:teuthology.orchestra.run.smithi143.stdout: "daemons": { 2024-01-16T15:02:05.558 INFO:teuthology.orchestra.run.smithi143.stdout: "summary": "", 2024-01-16T15:02:05.558 INFO:teuthology.orchestra.run.smithi143.stdout: "a": { 2024-01-16T15:02:05.558 INFO:teuthology.orchestra.run.smithi143.stdout: "start_epoch": 0, 2024-01-16T15:02:05.558 INFO:teuthology.orchestra.run.smithi143.stdout: "start_stamp": "0.000000", 2024-01-16T15:02:05.558 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 0, 2024-01-16T15:02:05.558 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "(unrecognized address family 0)/0", 2024-01-16T15:02:05.558 INFO:teuthology.orchestra.run.smithi143.stdout: "metadata": {}, 2024-01-16T15:02:05.558 INFO:teuthology.orchestra.run.smithi143.stdout: "task_status": {} 2024-01-16T15:02:05.558 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:02:05.558 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:02:05.559 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:02:05.559 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:02:05.559 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:02:05.559 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_events": {} 2024-01-16T15:02:05.559 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-16T15:02:05.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.574+0000 7fb989e00640 1 -- 172.21.15.143:0/192553059 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb95c06b4a0 msgr2=0x7fb95c06d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:05.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.574+0000 7fb989e00640 1 --2- 172.21.15.143:0/192553059 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb95c06b4a0 0x7fb95c06d960 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb974036000 tx=0x7fb974030000 comp rx=0 tx=0).stop 2024-01-16T15:02:05.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.574+0000 7fb989e00640 1 -- 172.21.15.143:0/192553059 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb984071d10 msgr2=0x7fb9840721a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:05.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.574+0000 7fb989e00640 1 --2- 172.21.15.143:0/192553059 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb984071d10 0x7fb9840721a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb9840d0040 tx=0x7fb97c00cb90 comp rx=0 tx=0).stop 2024-01-16T15:02:05.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.574+0000 7fb989e00640 1 -- 172.21.15.143:0/192553059 shutdown_connections 2024-01-16T15:02:05.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.574+0000 7fb989e00640 1 --2- 172.21.15.143:0/192553059 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb95c06b4a0 0x7fb95c06d960 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:05.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.574+0000 7fb989e00640 1 --2- 172.21.15.143:0/192553059 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb984071d10 0x7fb9840721a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:05.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.574+0000 7fb989e00640 1 --2- 172.21.15.143:0/192553059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb98415c510 0x7fb9840717d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:05.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.574+0000 7fb989e00640 1 --2- 172.21.15.143:0/192553059 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9640039c0 0x7fb98415bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:05.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.574+0000 7fb989e00640 1 -- 172.21.15.143:0/192553059 >> 172.21.15.143:0/192553059 conn(0x7fb98406a920 msgr2=0x7fb9840ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:05.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.575+0000 7fb989e00640 1 -- 172.21.15.143:0/192553059 shutdown_connections 2024-01-16T15:02:05.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.575+0000 7fb989e00640 1 -- 172.21.15.143:0/192553059 wait complete. 2024-01-16T15:02:05.573 INFO:tasks.cephfs.test_volumes:expected clone state = ('complete',) 2024-01-16T15:02:05.573 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-16T15:02:05.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.801+0000 7fbc36200640 1 Processor -- start 2024-01-16T15:02:05.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.802+0000 7fbc36200640 1 -- start start 2024-01-16T15:02:05.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.803+0000 7fbc36200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbc3006e160 0x7fbc300ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:05.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.803+0000 7fbc36200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbc300cd110 0x7fbc30070e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:05.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.803+0000 7fbc36200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc30161a10 con 0x7fbc30146eb0 2024-01-16T15:02:05.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.803+0000 7fbc36200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc300cd580 con 0x7fbc300cd110 2024-01-16T15:02:05.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.803+0000 7fbc36200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc30161cf0 con 0x7fbc3006e160 2024-01-16T15:02:05.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.804+0000 7fbc2fe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbc300cd110 0x7fbc30070e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:05.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.804+0000 7fbc2fe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbc300cd110 0x7fbc30070e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60842/0 (socket says 172.21.15.143:60842) 2024-01-16T15:02:05.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.804+0000 7fbc2fe00640 1 -- 172.21.15.143:0/2572962095 learned_addr learned my addr 172.21.15.143:0/2572962095 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:02:05.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.804+0000 7fbc26800640 1 --2- 172.21.15.143:0/2572962095 >> v2:172.21.15.204:3300/0 conn(0x7fbc3006e160 0x7fbc300ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:05.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.804+0000 7fbc2fe00640 1 -- 172.21.15.143:0/2572962095 >> v2:172.21.15.204:3300/0 conn(0x7fbc3006e160 msgr2=0x7fbc300ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:05.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.804+0000 7fbc2fe00640 1 --2- 172.21.15.143:0/2572962095 >> v2:172.21.15.204:3300/0 conn(0x7fbc3006e160 0x7fbc300ccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:05.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.804+0000 7fbc2fe00640 1 -- 172.21.15.143:0/2572962095 >> v1:172.21.15.204:6789/0 conn(0x7fbc30146eb0 legacy=0x7fbc30161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:05.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.804+0000 7fbc2fe00640 1 -- 172.21.15.143:0/2572962095 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc30161fd0 con 0x7fbc300cd110 2024-01-16T15:02:05.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.805+0000 7fbc2fe00640 1 --2- 172.21.15.143:0/2572962095 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc300cd110 0x7fbc30070e40 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fbc3006eab0 tx=0x7fbc2802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e297063c47dc3be7 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:05.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.806+0000 7fbc2ea00640 1 -- 172.21.15.143:0/2572962095 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2802ca80 con 0x7fbc300cd110 2024-01-16T15:02:05.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.806+0000 7fbc2ea00640 1 -- 172.21.15.143:0/2572962095 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc2802cbe0 con 0x7fbc300cd110 2024-01-16T15:02:05.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.806+0000 7fbc2ea00640 1 -- 172.21.15.143:0/2572962095 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc280343f0 con 0x7fbc300cd110 2024-01-16T15:02:05.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.806+0000 7fbc36200640 1 -- 172.21.15.143:0/2572962095 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc300cd110 msgr2=0x7fbc30070e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:05.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.806+0000 7fbc36200640 1 --2- 172.21.15.143:0/2572962095 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc300cd110 0x7fbc30070e40 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fbc3006eab0 tx=0x7fbc2802bee0 comp rx=0 tx=0).stop 2024-01-16T15:02:05.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.806+0000 7fbc36200640 1 -- 172.21.15.143:0/2572962095 shutdown_connections 2024-01-16T15:02:05.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.806+0000 7fbc36200640 1 --2- 172.21.15.143:0/2572962095 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc300cd110 0x7fbc30070e40 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:05.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.806+0000 7fbc36200640 1 --2- 172.21.15.143:0/2572962095 >> v2:172.21.15.204:3300/0 conn(0x7fbc3006e160 0x7fbc300ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:05.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.806+0000 7fbc36200640 1 -- 172.21.15.143:0/2572962095 >> 172.21.15.143:0/2572962095 conn(0x7fbc300b0c50 msgr2=0x7fbc300b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:05.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.806+0000 7fbc36200640 1 -- 172.21.15.143:0/2572962095 shutdown_connections 2024-01-16T15:02:05.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.806+0000 7fbc36200640 1 -- 172.21.15.143:0/2572962095 wait complete. 2024-01-16T15:02:05.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.808+0000 7fbc36200640 1 Processor -- start 2024-01-16T15:02:05.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.808+0000 7fbc36200640 1 -- start start 2024-01-16T15:02:05.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.809+0000 7fbc36200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbc3006e160 0x7fbc301e88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:05.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.809+0000 7fbc36200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc300cd110 0x7fbc300ad1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:05.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.809+0000 7fbc36200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc30146eb0 0x7fbc301e6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:05.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.809+0000 7fbc36200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3009b620 con 0x7fbc300cd110 2024-01-16T15:02:05.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.809+0000 7fbc36200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3015db40 con 0x7fbc30146eb0 2024-01-16T15:02:05.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.809+0000 7fbc36200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc301e8de0 con 0x7fbc3006e160 2024-01-16T15:02:05.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.809+0000 7fbc26800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc300cd110 0x7fbc300ad1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:05.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.809+0000 7fbc26800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc300cd110 0x7fbc300ad1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60852/0 (socket says 172.21.15.143:60852) 2024-01-16T15:02:05.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.809+0000 7fbc26800640 1 -- 172.21.15.143:0/3170596714 learned_addr learned my addr 172.21.15.143:0/3170596714 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:02:05.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.810+0000 7fbc26800640 1 -- 172.21.15.143:0/3170596714 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbc3006e160 msgr2=0x7fbc301e88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:05.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.810+0000 7fbc26800640 1 --2- 172.21.15.143:0/3170596714 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbc3006e160 0x7fbc301e88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:05.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.810+0000 7fbc26800640 1 -- 172.21.15.143:0/3170596714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc30146eb0 msgr2=0x7fbc301e6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:05.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.810+0000 7fbc2fe00640 1 --2- 172.21.15.143:0/3170596714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc30146eb0 0x7fbc301e6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:05.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.810+0000 7fbc26800640 1 --2- 172.21.15.143:0/3170596714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc30146eb0 0x7fbc301e6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:05.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.810+0000 7fbc26800640 1 -- 172.21.15.143:0/3170596714 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc28009b30 con 0x7fbc300cd110 2024-01-16T15:02:05.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.810+0000 7fbc26800640 1 --2- 172.21.15.143:0/3170596714 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc300cd110 0x7fbc300ad1f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fbc1800d800 tx=0x7fbc1800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:05.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.810+0000 7fbc27e00640 1 -- 172.21.15.143:0/3170596714 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1800b6c0 con 0x7fbc300cd110 2024-01-16T15:02:05.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.810+0000 7fbc27e00640 1 -- 172.21.15.143:0/3170596714 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc1800b820 con 0x7fbc300cd110 2024-01-16T15:02:05.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.810+0000 7fbc36200640 1 -- 172.21.15.143:0/3170596714 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc3009a9a0 con 0x7fbc300cd110 2024-01-16T15:02:05.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.810+0000 7fbc27e00640 1 -- 172.21.15.143:0/3170596714 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1800bbb0 con 0x7fbc300cd110 2024-01-16T15:02:05.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.811+0000 7fbc36200640 1 -- 172.21.15.143:0/3170596714 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc3009ad30 con 0x7fbc300cd110 2024-01-16T15:02:05.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.812+0000 7fbc36200640 1 -- 172.21.15.143:0/3170596714 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc3004e8b0 con 0x7fbc300cd110 2024-01-16T15:02:05.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.813+0000 7fbc27e00640 1 -- 172.21.15.143:0/3170596714 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7fbc1800bd10 con 0x7fbc300cd110 2024-01-16T15:02:05.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.813+0000 7fbc27e00640 1 --2- 172.21.15.143:0/3170596714 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbc0806b470 0x7fbc0806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:05.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.814+0000 7fbc27e00640 1 -- 172.21.15.143:0/3170596714 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc1808a5d0 con 0x7fbc300cd110 2024-01-16T15:02:05.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.814+0000 7fbc2f400640 1 --2- 172.21.15.143:0/3170596714 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbc0806b470 0x7fbc0806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:05.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.814+0000 7fbc2f400640 1 --2- 172.21.15.143:0/3170596714 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbc0806b470 0x7fbc0806d930 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbc20006fd0 tx=0x7fbc20006bc0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:05.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:05.818+0000 7fbc27e00640 1 -- 172.21.15.143:0/3170596714 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbc18055090 con 0x7fbc300cd110 2024-01-16T15:02:06.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:06.043+0000 7fbc36200640 1 -- 172.21.15.143:0/3170596714 --> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc30147290 con 0x7fbc0806b470 2024-01-16T15:02:06.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:06.052+0000 7fbc27e00640 1 -- 172.21.15.143:0/3170596714 <== mgr.4113 v2:172.21.15.143:6832/3153492115 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+344 (secure 0 0 0) 0x7fbc20009280 con 0x7fbc0806b470 2024-01-16T15:02:06.032 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-16T15:02:06.032 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-16T15:02:06.032 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-16T15:02:06.033 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-16T15:02:06.033 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-16T15:02:06.033 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-16T15:02:06.033 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-16T15:02:06.033 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:02:06.033 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-16T15:02:06.033 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 0.0, 2024-01-16T15:02:06.033 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "0 B/3 GB", 2024-01-16T15:02:06.033 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "0/3", 2024-01-16T15:02:06.033 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-16T15:02:06.033 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-16T15:02:06.033 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:02:06.033 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:02:06.033 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-16T15:02:06.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:06.054+0000 7fbc36200640 1 -- 172.21.15.143:0/3170596714 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbc0806b470 msgr2=0x7fbc0806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:06.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:06.054+0000 7fbc36200640 1 --2- 172.21.15.143:0/3170596714 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbc0806b470 0x7fbc0806d930 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbc20006fd0 tx=0x7fbc20006bc0 comp rx=0 tx=0).stop 2024-01-16T15:02:06.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:06.054+0000 7fbc36200640 1 -- 172.21.15.143:0/3170596714 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc300cd110 msgr2=0x7fbc300ad1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:06.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:06.054+0000 7fbc36200640 1 --2- 172.21.15.143:0/3170596714 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc300cd110 0x7fbc300ad1f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fbc1800d800 tx=0x7fbc1800dcc0 comp rx=0 tx=0).stop 2024-01-16T15:02:06.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:06.055+0000 7fbc36200640 1 -- 172.21.15.143:0/3170596714 shutdown_connections 2024-01-16T15:02:06.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:06.055+0000 7fbc36200640 1 --2- 172.21.15.143:0/3170596714 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbc0806b470 0x7fbc0806d930 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:06.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:06.055+0000 7fbc36200640 1 --2- 172.21.15.143:0/3170596714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc30146eb0 0x7fbc301e6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:06.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:06.055+0000 7fbc36200640 1 --2- 172.21.15.143:0/3170596714 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc300cd110 0x7fbc300ad1f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:06.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:06.055+0000 7fbc36200640 1 --2- 172.21.15.143:0/3170596714 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbc3006e160 0x7fbc301e88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:06.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:06.055+0000 7fbc36200640 1 -- 172.21.15.143:0/3170596714 >> 172.21.15.143:0/3170596714 conn(0x7fbc300b0c50 msgr2=0x7fbc30144cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:06.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:06.055+0000 7fbc36200640 1 -- 172.21.15.143:0/3170596714 shutdown_connections 2024-01-16T15:02:06.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:06.055+0000 7fbc36200640 1 -- 172.21.15.143:0/3170596714 wait complete. 2024-01-16T15:02:06.052 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-16T15:02:07.053 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-16T15:02:07.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.299+0000 7f7d4da00640 1 Processor -- start 2024-01-16T15:02:07.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.300+0000 7f7d4da00640 1 -- start start 2024-01-16T15:02:07.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.302+0000 7f7d4da00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7d480cc500 0x7f7d480cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:07.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.302+0000 7f7d4da00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7d480c3ec0 0x7f7d480cd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:07.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.302+0000 7f7d4da00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d480705c0 con 0x7f7d481595c0 2024-01-16T15:02:07.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.303+0000 7f7d4da00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d48071140 con 0x7f7d480cc500 2024-01-16T15:02:07.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.303+0000 7f7d4da00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d480cd880 con 0x7f7d480c3ec0 2024-01-16T15:02:07.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.303+0000 7f7d46000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7d480cc500 0x7f7d480cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:07.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.303+0000 7f7d46000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7d480cc500 0x7f7d480cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60864/0 (socket says 172.21.15.143:60864) 2024-01-16T15:02:07.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.303+0000 7f7d46000640 1 -- 172.21.15.143:0/740904039 learned_addr learned my addr 172.21.15.143:0/740904039 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:02:07.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.303+0000 7f7d47400640 1 --2- 172.21.15.143:0/740904039 >> v2:172.21.15.204:3300/0 conn(0x7f7d480c3ec0 0x7f7d480cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:07.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.304+0000 7f7d46000640 1 -- 172.21.15.143:0/740904039 >> v2:172.21.15.204:3300/0 conn(0x7f7d480c3ec0 msgr2=0x7f7d480cd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:07.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.304+0000 7f7d46000640 1 --2- 172.21.15.143:0/740904039 >> v2:172.21.15.204:3300/0 conn(0x7f7d480c3ec0 0x7f7d480cd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:07.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.304+0000 7f7d46000640 1 -- 172.21.15.143:0/740904039 >> v1:172.21.15.204:6789/0 conn(0x7f7d481595c0 legacy=0x7f7d480cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:07.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.304+0000 7f7d46000640 1 -- 172.21.15.143:0/740904039 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d48070810 con 0x7f7d480cc500 2024-01-16T15:02:07.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.306+0000 7f7d46000640 1 --2- 172.21.15.143:0/740904039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d480cc500 0x7f7d480cce00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7d480c5fc0 tx=0x7f7d3802d290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eff03471c6494735 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:07.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.306+0000 7f7d45600640 1 -- 172.21.15.143:0/740904039 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3800a2d0 con 0x7f7d480cc500 2024-01-16T15:02:07.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.306+0000 7f7d45600640 1 -- 172.21.15.143:0/740904039 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d3800a430 con 0x7f7d480cc500 2024-01-16T15:02:07.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.307+0000 7f7d45600640 1 -- 172.21.15.143:0/740904039 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3800a7c0 con 0x7f7d480cc500 2024-01-16T15:02:07.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.307+0000 7f7d4da00640 1 -- 172.21.15.143:0/740904039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d480cc500 msgr2=0x7f7d480cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:07.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.307+0000 7f7d4da00640 1 --2- 172.21.15.143:0/740904039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d480cc500 0x7f7d480cce00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7d480c5fc0 tx=0x7f7d3802d290 comp rx=0 tx=0).stop 2024-01-16T15:02:07.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.308+0000 7f7d4da00640 1 -- 172.21.15.143:0/740904039 shutdown_connections 2024-01-16T15:02:07.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.308+0000 7f7d4da00640 1 --2- 172.21.15.143:0/740904039 >> v2:172.21.15.204:3300/0 conn(0x7f7d480c3ec0 0x7f7d480cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:07.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.308+0000 7f7d4da00640 1 --2- 172.21.15.143:0/740904039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d480cc500 0x7f7d480cce00 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:07.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.308+0000 7f7d4da00640 1 -- 172.21.15.143:0/740904039 >> 172.21.15.143:0/740904039 conn(0x7f7d480b0c50 msgr2=0x7f7d480b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:07.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.308+0000 7f7d4da00640 1 -- 172.21.15.143:0/740904039 shutdown_connections 2024-01-16T15:02:07.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.308+0000 7f7d4da00640 1 -- 172.21.15.143:0/740904039 wait complete. 2024-01-16T15:02:07.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.310+0000 7f7d4da00640 1 Processor -- start 2024-01-16T15:02:07.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.311+0000 7f7d4da00640 1 -- start start 2024-01-16T15:02:07.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.312+0000 7f7d4da00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d480c3ec0 0x7f7d48153ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:07.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.312+0000 7f7d4da00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d480cc500 0x7f7d480b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:07.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.312+0000 7f7d4da00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d481595c0 0x7f7d480ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:07.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.312+0000 7f7d4da00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d481543e0 con 0x7f7d481595c0 2024-01-16T15:02:07.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.312+0000 7f7d4da00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d480b2690 con 0x7f7d480c3ec0 2024-01-16T15:02:07.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.312+0000 7f7d4da00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d480b2970 con 0x7f7d480cc500 2024-01-16T15:02:07.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.312+0000 7f7d47400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d481595c0 0x7f7d480ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:07.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.312+0000 7f7d47400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d481595c0 0x7f7d480ca2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60880/0 (socket says 172.21.15.143:60880) 2024-01-16T15:02:07.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.312+0000 7f7d47400640 1 -- 172.21.15.143:0/1128979310 learned_addr learned my addr 172.21.15.143:0/1128979310 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:02:07.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.313+0000 7f7d46a00640 1 --2- 172.21.15.143:0/1128979310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d480c3ec0 0x7f7d48153ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:07.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.313+0000 7f7d47400640 1 -- 172.21.15.143:0/1128979310 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d480cc500 msgr2=0x7f7d480b1d20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-16T15:02:07.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.313+0000 7f7d47400640 1 --2- 172.21.15.143:0/1128979310 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d480cc500 0x7f7d480b1d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:07.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.313+0000 7f7d47400640 1 -- 172.21.15.143:0/1128979310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d480c3ec0 msgr2=0x7f7d48153ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:07.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.313+0000 7f7d47400640 1 --2- 172.21.15.143:0/1128979310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d480c3ec0 0x7f7d48153ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:07.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.313+0000 7f7d47400640 1 -- 172.21.15.143:0/1128979310 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d3800c050 con 0x7f7d481595c0 2024-01-16T15:02:07.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.313+0000 7f7d47400640 1 --2- 172.21.15.143:0/1128979310 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d481595c0 0x7f7d480ca2e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7d4000d800 tx=0x7f7d4000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:07.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.314+0000 7f7d3ea00640 1 -- 172.21.15.143:0/1128979310 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4000b6c0 con 0x7f7d481595c0 2024-01-16T15:02:07.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.314+0000 7f7d3ea00640 1 -- 172.21.15.143:0/1128979310 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d4000b820 con 0x7f7d481595c0 2024-01-16T15:02:07.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.314+0000 7f7d3ea00640 1 -- 172.21.15.143:0/1128979310 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4000bbd0 con 0x7f7d481595c0 2024-01-16T15:02:07.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.315+0000 7f7d4da00640 1 -- 172.21.15.143:0/1128979310 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d480b2c50 con 0x7f7d481595c0 2024-01-16T15:02:07.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.316+0000 7f7d4da00640 1 -- 172.21.15.143:0/1128979310 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d480b3210 con 0x7f7d481595c0 2024-01-16T15:02:07.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.318+0000 7f7d4da00640 1 -- 172.21.15.143:0/1128979310 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d480b3400 con 0x7f7d481595c0 2024-01-16T15:02:07.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.318+0000 7f7d3ea00640 1 -- 172.21.15.143:0/1128979310 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7f7d4000bd30 con 0x7f7d481595c0 2024-01-16T15:02:07.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.320+0000 7f7d3ea00640 1 --2- 172.21.15.143:0/1128979310 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f7d2006b4c0 0x7f7d2006d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:07.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.320+0000 7f7d3ea00640 1 -- 172.21.15.143:0/1128979310 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7d4008a300 con 0x7f7d481595c0 2024-01-16T15:02:07.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.324+0000 7f7d3ea00640 1 -- 172.21.15.143:0/1128979310 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7d40055060 con 0x7f7d481595c0 2024-01-16T15:02:07.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.327+0000 7f7d46a00640 1 --2- 172.21.15.143:0/1128979310 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f7d2006b4c0 0x7f7d2006d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:07.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.329+0000 7f7d46a00640 1 --2- 172.21.15.143:0/1128979310 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f7d2006b4c0 0x7f7d2006d980 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7d30006fd0 tx=0x7f7d30008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:07.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.668+0000 7f7d4da00640 1 -- 172.21.15.143:0/1128979310 --> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d48070dd0 con 0x7f7d2006b4c0 2024-01-16T15:02:07.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.689+0000 7f7d3ea00640 1 -- 172.21.15.143:0/1128979310 <== mgr.4113 v2:172.21.15.143:6832/3153492115 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+344 (secure 0 0 0) 0x7f7d3000bd80 con 0x7f7d2006b4c0 2024-01-16T15:02:07.670 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-16T15:02:07.670 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-16T15:02:07.670 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-16T15:02:07.670 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-16T15:02:07.670 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-16T15:02:07.670 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-16T15:02:07.670 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-16T15:02:07.670 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:02:07.670 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-16T15:02:07.671 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 0.0, 2024-01-16T15:02:07.671 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "0 B/3 GB", 2024-01-16T15:02:07.671 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "0/3", 2024-01-16T15:02:07.671 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-16T15:02:07.671 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-16T15:02:07.671 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:02:07.671 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:02:07.671 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-16T15:02:07.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.693+0000 7f7d4da00640 1 -- 172.21.15.143:0/1128979310 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f7d2006b4c0 msgr2=0x7f7d2006d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:07.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.693+0000 7f7d4da00640 1 --2- 172.21.15.143:0/1128979310 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f7d2006b4c0 0x7f7d2006d980 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7d30006fd0 tx=0x7f7d30008040 comp rx=0 tx=0).stop 2024-01-16T15:02:07.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.693+0000 7f7d4da00640 1 -- 172.21.15.143:0/1128979310 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d481595c0 msgr2=0x7f7d480ca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:07.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.693+0000 7f7d4da00640 1 --2- 172.21.15.143:0/1128979310 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d481595c0 0x7f7d480ca2e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7d4000d800 tx=0x7f7d4000dcc0 comp rx=0 tx=0).stop 2024-01-16T15:02:07.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.693+0000 7f7d4da00640 1 -- 172.21.15.143:0/1128979310 shutdown_connections 2024-01-16T15:02:07.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.693+0000 7f7d4da00640 1 --2- 172.21.15.143:0/1128979310 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f7d2006b4c0 0x7f7d2006d980 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:07.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.693+0000 7f7d4da00640 1 --2- 172.21.15.143:0/1128979310 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d481595c0 0x7f7d480ca2e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:07.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.693+0000 7f7d4da00640 1 --2- 172.21.15.143:0/1128979310 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d480cc500 0x7f7d480b1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:07.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.693+0000 7f7d4da00640 1 --2- 172.21.15.143:0/1128979310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d480c3ec0 0x7f7d48153ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:07.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.693+0000 7f7d4da00640 1 -- 172.21.15.143:0/1128979310 >> 172.21.15.143:0/1128979310 conn(0x7f7d480b0c50 msgr2=0x7f7d480cf990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:07.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.693+0000 7f7d4da00640 1 -- 172.21.15.143:0/1128979310 shutdown_connections 2024-01-16T15:02:07.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:07.693+0000 7f7d4da00640 1 -- 172.21.15.143:0/1128979310 wait complete. 2024-01-16T15:02:07.696 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-16T15:02:08.697 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-16T15:02:09.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.055+0000 7f0110e00640 1 Processor -- start 2024-01-16T15:02:09.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.056+0000 7f0110e00640 1 -- start start 2024-01-16T15:02:09.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.058+0000 7f0110e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f010c0ccbd0 0x7f010c1660b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:09.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.058+0000 7f0110e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f010c0c9650 0x7f010c0c9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:09.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.058+0000 7f0110e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f010c1655f0 con 0x7f010c0ccfb0 2024-01-16T15:02:09.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.058+0000 7f0110e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f010c0cc380 con 0x7f010c0ccbd0 2024-01-16T15:02:09.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.058+0000 7f0110e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f010c162b30 con 0x7f010c0c9650 2024-01-16T15:02:09.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.059+0000 7f0109e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f010c0ccbd0 0x7f010c1660b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:09.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.059+0000 7f0109e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f010c0ccbd0 0x7f010c1660b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60892/0 (socket says 172.21.15.143:60892) 2024-01-16T15:02:09.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.059+0000 7f0109e00640 1 -- 172.21.15.143:0/2500366086 learned_addr learned my addr 172.21.15.143:0/2500366086 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:02:09.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.060+0000 7f0109e00640 1 -- 172.21.15.143:0/2500366086 >> v2:172.21.15.204:3300/0 conn(0x7f010c0c9650 msgr2=0x7f010c0c9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:09.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.060+0000 7f010a800640 1 --2- 172.21.15.143:0/2500366086 >> v2:172.21.15.204:3300/0 conn(0x7f010c0c9650 0x7f010c0c9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:09.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.060+0000 7f0109e00640 1 --2- 172.21.15.143:0/2500366086 >> v2:172.21.15.204:3300/0 conn(0x7f010c0c9650 0x7f010c0c9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:09.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.060+0000 7f0109e00640 1 -- 172.21.15.143:0/2500366086 >> v1:172.21.15.204:6789/0 conn(0x7f010c0ccfb0 legacy=0x7f010c162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:09.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.060+0000 7f0109e00640 1 -- 172.21.15.143:0/2500366086 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f010c162e10 con 0x7f010c0ccbd0 2024-01-16T15:02:09.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.061+0000 7f010a800640 1 --2- 172.21.15.143:0/2500366086 >> v2:172.21.15.204:3300/0 conn(0x7f010c0c9650 0x7f010c0c9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:02:09.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.063+0000 7f0109e00640 1 --2- 172.21.15.143:0/2500366086 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f010c0ccbd0 0x7f010c1660b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f010c0c5fe0 tx=0x7f00f402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdd736ac0132a1a3 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:09.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.063+0000 7f0108a00640 1 -- 172.21.15.143:0/2500366086 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f4031870 con 0x7f010c0ccbd0 2024-01-16T15:02:09.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.063+0000 7f0108a00640 1 -- 172.21.15.143:0/2500366086 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f40319d0 con 0x7f010c0ccbd0 2024-01-16T15:02:09.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.063+0000 7f0108a00640 1 -- 172.21.15.143:0/2500366086 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f4031d60 con 0x7f010c0ccbd0 2024-01-16T15:02:09.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.065+0000 7f0110e00640 1 -- 172.21.15.143:0/2500366086 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f010c0ccbd0 msgr2=0x7f010c1660b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:09.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.065+0000 7f0110e00640 1 --2- 172.21.15.143:0/2500366086 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f010c0ccbd0 0x7f010c1660b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f010c0c5fe0 tx=0x7f00f402b910 comp rx=0 tx=0).stop 2024-01-16T15:02:09.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.066+0000 7f0110e00640 1 -- 172.21.15.143:0/2500366086 shutdown_connections 2024-01-16T15:02:09.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.066+0000 7f0110e00640 1 --2- 172.21.15.143:0/2500366086 >> v2:172.21.15.204:3300/0 conn(0x7f010c0c9650 0x7f010c0c9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:09.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.066+0000 7f0110e00640 1 --2- 172.21.15.143:0/2500366086 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f010c0ccbd0 0x7f010c1660b0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:09.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.066+0000 7f0110e00640 1 -- 172.21.15.143:0/2500366086 >> 172.21.15.143:0/2500366086 conn(0x7f010c0b0c50 msgr2=0x7f010c0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:09.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.066+0000 7f0110e00640 1 -- 172.21.15.143:0/2500366086 shutdown_connections 2024-01-16T15:02:09.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.066+0000 7f0110e00640 1 -- 172.21.15.143:0/2500366086 wait complete. 2024-01-16T15:02:09.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.069+0000 7f0110e00640 1 Processor -- start 2024-01-16T15:02:09.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.070+0000 7f0110e00640 1 -- start start 2024-01-16T15:02:09.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.072+0000 7f0110e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f010c0c9650 0x7f010c164930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:09.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.072+0000 7f0110e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f010c0ccbd0 0x7f010c15f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:09.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.072+0000 7f0110e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f010c0ccfb0 0x7f010c0adb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:09.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.072+0000 7f0110e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c1edfb0 con 0x7f010c0ccbd0 2024-01-16T15:02:09.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.072+0000 7f0110e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c1ed3e0 con 0x7f010c0ccfb0 2024-01-16T15:02:09.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.072+0000 7f0110e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f010c164f10 con 0x7f010c0c9650 2024-01-16T15:02:09.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.072+0000 7f0109400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f010c0ccbd0 0x7f010c15f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:09.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.072+0000 7f0109400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f010c0ccbd0 0x7f010c15f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60904/0 (socket says 172.21.15.143:60904) 2024-01-16T15:02:09.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.072+0000 7f0109400640 1 -- 172.21.15.143:0/2503514451 learned_addr learned my addr 172.21.15.143:0/2503514451 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:02:09.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.073+0000 7f0109e00640 1 --2- 172.21.15.143:0/2503514451 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f010c0c9650 0x7f010c164930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:09.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.073+0000 7f0109400640 1 -- 172.21.15.143:0/2503514451 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f010c0c9650 msgr2=0x7f010c164930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:09.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.073+0000 7f0109400640 1 --2- 172.21.15.143:0/2503514451 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f010c0c9650 0x7f010c164930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:09.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.073+0000 7f0109400640 1 -- 172.21.15.143:0/2503514451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f010c0ccfb0 msgr2=0x7f010c0adb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:09.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.073+0000 7f0109400640 1 --2- 172.21.15.143:0/2503514451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f010c0ccfb0 0x7f010c0adb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:09.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.073+0000 7f0109400640 1 -- 172.21.15.143:0/2503514451 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f40094d0 con 0x7f010c0ccbd0 2024-01-16T15:02:09.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.074+0000 7f0109400640 1 --2- 172.21.15.143:0/2503514451 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f010c0ccbd0 0x7f010c15f1a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00c430 tx=0x7f00fc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:09.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.075+0000 7f0102000640 1 -- 172.21.15.143:0/2503514451 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc011790 con 0x7f010c0ccbd0 2024-01-16T15:02:09.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.075+0000 7f0102000640 1 -- 172.21.15.143:0/2503514451 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00fc0118f0 con 0x7f010c0ccbd0 2024-01-16T15:02:09.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.075+0000 7f0102000640 1 -- 172.21.15.143:0/2503514451 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc011ce0 con 0x7f010c0ccbd0 2024-01-16T15:02:09.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.075+0000 7f0110e00640 1 -- 172.21.15.143:0/2503514451 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f010c1ed610 con 0x7f010c0ccbd0 2024-01-16T15:02:09.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.075+0000 7f0110e00640 1 -- 172.21.15.143:0/2503514451 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f010c1edbd0 con 0x7f010c0ccbd0 2024-01-16T15:02:09.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.077+0000 7f0110e00640 1 -- 172.21.15.143:0/2503514451 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f010c1ee0f0 con 0x7f010c0ccbd0 2024-01-16T15:02:09.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.078+0000 7f0102000640 1 -- 172.21.15.143:0/2503514451 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7f00fc01c460 con 0x7f010c0ccbd0 2024-01-16T15:02:09.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.083+0000 7f0102000640 1 --2- 172.21.15.143:0/2503514451 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f00e406b470 0x7f00e406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:09.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.083+0000 7f0102000640 1 -- 172.21.15.143:0/2503514451 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f00fc089d80 con 0x7f010c0ccbd0 2024-01-16T15:02:09.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.085+0000 7f0109e00640 1 --2- 172.21.15.143:0/2503514451 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f00e406b470 0x7f00e406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:09.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.087+0000 7f0109e00640 1 --2- 172.21.15.143:0/2503514451 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f00e406b470 0x7f00e406d930 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f00f4036000 tx=0x7f00f403c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:09.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.089+0000 7f0102000640 1 -- 172.21.15.143:0/2503514451 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f010c1ee0f0 con 0x7f010c0ccbd0 2024-01-16T15:02:09.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.497+0000 7f0110e00640 1 -- 172.21.15.143:0/2503514451 --> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f010c0cd410 con 0x7f00e406b470 2024-01-16T15:02:09.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.503+0000 7f0102000640 1 -- 172.21.15.143:0/2503514451 <== mgr.4113 v2:172.21.15.143:6832/3153492115 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7f00f4031c40 con 0x7f00e406b470 2024-01-16T15:02:09.484 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-16T15:02:09.484 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-16T15:02:09.484 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-16T15:02:09.484 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-16T15:02:09.484 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-16T15:02:09.484 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-16T15:02:09.484 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-16T15:02:09.484 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:02:09.484 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-16T15:02:09.485 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 33.33333333333333, 2024-01-16T15:02:09.485 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "1 GB/3 GB", 2024-01-16T15:02:09.485 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "1/3", 2024-01-16T15:02:09.485 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-16T15:02:09.485 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-16T15:02:09.485 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:02:09.485 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:02:09.485 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-16T15:02:09.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.505+0000 7f0110e00640 1 -- 172.21.15.143:0/2503514451 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f00e406b470 msgr2=0x7f00e406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:09.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.505+0000 7f0110e00640 1 --2- 172.21.15.143:0/2503514451 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f00e406b470 0x7f00e406d930 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f00f4036000 tx=0x7f00f403c040 comp rx=0 tx=0).stop 2024-01-16T15:02:09.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.505+0000 7f0110e00640 1 -- 172.21.15.143:0/2503514451 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f010c0ccbd0 msgr2=0x7f010c15f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:09.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.505+0000 7f0110e00640 1 --2- 172.21.15.143:0/2503514451 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f010c0ccbd0 0x7f010c15f1a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00c430 tx=0x7f00fc00c8f0 comp rx=0 tx=0).stop 2024-01-16T15:02:09.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.505+0000 7f0110e00640 1 -- 172.21.15.143:0/2503514451 shutdown_connections 2024-01-16T15:02:09.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.505+0000 7f0110e00640 1 --2- 172.21.15.143:0/2503514451 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f00e406b470 0x7f00e406d930 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:09.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.505+0000 7f0110e00640 1 --2- 172.21.15.143:0/2503514451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f010c0ccfb0 0x7f010c0adb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:09.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.505+0000 7f0110e00640 1 --2- 172.21.15.143:0/2503514451 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f010c0ccbd0 0x7f010c15f1a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:09.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.505+0000 7f0110e00640 1 --2- 172.21.15.143:0/2503514451 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f010c0c9650 0x7f010c164930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:09.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.505+0000 7f0110e00640 1 -- 172.21.15.143:0/2503514451 >> 172.21.15.143:0/2503514451 conn(0x7f010c0b0c50 msgr2=0x7f010c06c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:09.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.506+0000 7f0110e00640 1 -- 172.21.15.143:0/2503514451 shutdown_connections 2024-01-16T15:02:09.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:09.506+0000 7f0110e00640 1 -- 172.21.15.143:0/2503514451 wait complete. 2024-01-16T15:02:09.502 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-16T15:02:10.503 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-16T15:02:10.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.742+0000 7f36ebe00640 1 Processor -- start 2024-01-16T15:02:10.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.742+0000 7f36ebe00640 1 -- start start 2024-01-16T15:02:10.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.744+0000 7f36ebe00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f36ec15a1c0 0x7f36ec06d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:10.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.744+0000 7f36ebe00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36ec06d430 con 0x7f36ec0b0b40 2024-01-16T15:02:10.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.744+0000 7f36ebe00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36ec15ec50 con 0x7f36ec15e5e0 2024-01-16T15:02:10.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.744+0000 7f36ebe00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36ec15ea70 con 0x7f36ec15a1c0 2024-01-16T15:02:10.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.744+0000 7f36e3e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f36ec0b0b40 0x7f36ec15ded0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:38238/0 (socket says 172.21.15.143:38238) 2024-01-16T15:02:10.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.744+0000 7f36e3e00640 1 -- 172.21.15.143:0/367846978 learned_addr learned my addr 172.21.15.143:0/367846978 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:02:10.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.744+0000 7f36eaa00640 1 --2- 172.21.15.143:0/367846978 >> v2:172.21.15.204:3300/0 conn(0x7f36ec15a1c0 0x7f36ec06d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:10.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.745+0000 7f36ea000640 1 -- 172.21.15.143:0/367846978 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3089969596 0 0) 0x7f36ec06d430 con 0x7f36ec0b0b40 2024-01-16T15:02:10.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.745+0000 7f36ea000640 1 -- 172.21.15.143:0/367846978 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36cc003520 con 0x7f36ec0b0b40 2024-01-16T15:02:10.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.745+0000 7f36ea000640 1 -- 172.21.15.143:0/367846978 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3069945560 0 0) 0x7f36cc003520 con 0x7f36ec0b0b40 2024-01-16T15:02:10.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.745+0000 7f36ea000640 1 -- 172.21.15.143:0/367846978 >> v2:172.21.15.204:3300/0 conn(0x7f36ec15a1c0 msgr2=0x7f36ec06d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:10.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.745+0000 7f36ea000640 1 --2- 172.21.15.143:0/367846978 >> v2:172.21.15.204:3300/0 conn(0x7f36ec15a1c0 0x7f36ec06d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:10.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.745+0000 7f36ea000640 1 -- 172.21.15.143:0/367846978 >> v1:172.21.15.204:6789/0 conn(0x7f36ec15e5e0 legacy=0x7f36ec070e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:10.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.745+0000 7f36ea000640 1 -- 172.21.15.143:0/367846978 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ec06cb00 con 0x7f36ec0b0b40 2024-01-16T15:02:10.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.745+0000 7f36ea000640 1 -- 172.21.15.143:0/367846978 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7f36d4001670 con 0x7f36ec0b0b40 2024-01-16T15:02:10.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.745+0000 7f36ea000640 1 -- 172.21.15.143:0/367846978 >> v1:172.21.15.143:6789/0 conn(0x7f36ec0b0b40 legacy=0x7f36ec15ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:10.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.745+0000 7f36ea000640 1 --2- 172.21.15.143:0/367846978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36cc0039c0 0x7f36cc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:10.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.745+0000 7f36ea000640 1 --2- 172.21.15.143:0/367846978 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36cc0047c0 0x7f36cc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:10.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.745+0000 7f36ea000640 1 --2- 172.21.15.143:0/367846978 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36cc005180 0x7f36cc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:10.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.745+0000 7f36ea000640 1 -- 172.21.15.143:0/367846978 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec15ea70 con 0x7f36cc0047c0 2024-01-16T15:02:10.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.745+0000 7f36ea000640 1 -- 172.21.15.143:0/367846978 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36cc00c080 con 0x7f36cc0039c0 2024-01-16T15:02:10.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.745+0000 7f36ea000640 1 -- 172.21.15.143:0/367846978 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36cc00c240 con 0x7f36cc005180 2024-01-16T15:02:10.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.745+0000 7f36e3e00640 1 --2- 172.21.15.143:0/367846978 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36cc0047c0 0x7f36cc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:10.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.746+0000 7f36e3e00640 1 -- 172.21.15.143:0/367846978 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36cc005180 msgr2=0x7f36cc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:10.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.746+0000 7f36eaa00640 1 --2- 172.21.15.143:0/367846978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36cc0039c0 0x7f36cc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:10.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.746+0000 7f36eb400640 1 --2- 172.21.15.143:0/367846978 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36cc005180 0x7f36cc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:10.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.746+0000 7f36e3e00640 1 --2- 172.21.15.143:0/367846978 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36cc005180 0x7f36cc00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:10.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.746+0000 7f36e3e00640 1 -- 172.21.15.143:0/367846978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36cc0039c0 msgr2=0x7f36cc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:10.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.746+0000 7f36e3e00640 1 --2- 172.21.15.143:0/367846978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36cc0039c0 0x7f36cc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:10.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.746+0000 7f36e3e00640 1 -- 172.21.15.143:0/367846978 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36cc003520 con 0x7f36cc0047c0 2024-01-16T15:02:10.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.746+0000 7f36eb400640 1 --2- 172.21.15.143:0/367846978 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36cc005180 0x7f36cc00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:02:10.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.746+0000 7f36eaa00640 1 --2- 172.21.15.143:0/367846978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36cc0039c0 0x7f36cc003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:02:10.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.746+0000 7f36eb400640 1 -- 172.21.15.143:0/367846978 reap_dead start 2024-01-16T15:02:10.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.747+0000 7f36e3e00640 1 --2- 172.21.15.143:0/367846978 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36cc0047c0 0x7f36cc004c40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f36d400c0e0 tx=0x7f36cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdedc05590297177 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:10.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.747+0000 7f36ea000640 1 -- 172.21.15.143:0/367846978 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d4005ad0 con 0x7f36cc0047c0 2024-01-16T15:02:10.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.747+0000 7f36ea000640 1 -- 172.21.15.143:0/367846978 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d4005c30 con 0x7f36cc0047c0 2024-01-16T15:02:10.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.747+0000 7f36ea000640 1 -- 172.21.15.143:0/367846978 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d402ca50 con 0x7f36cc0047c0 2024-01-16T15:02:10.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.748+0000 7f36ebe00640 1 -- 172.21.15.143:0/367846978 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36cc0047c0 msgr2=0x7f36cc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:10.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.748+0000 7f36ebe00640 1 --2- 172.21.15.143:0/367846978 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36cc0047c0 0x7f36cc004c40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f36d400c0e0 tx=0x7f36cc0057f0 comp rx=0 tx=0).stop 2024-01-16T15:02:10.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.748+0000 7f36ebe00640 1 -- 172.21.15.143:0/367846978 shutdown_connections 2024-01-16T15:02:10.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.748+0000 7f36ebe00640 1 --2- 172.21.15.143:0/367846978 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36cc0047c0 0x7f36cc004c40 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:10.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.748+0000 7f36ebe00640 1 -- 172.21.15.143:0/367846978 >> 172.21.15.143:0/367846978 conn(0x7f36ec0cbb40 msgr2=0x7f36ec0cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:10.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.748+0000 7f36ebe00640 1 -- 172.21.15.143:0/367846978 shutdown_connections 2024-01-16T15:02:10.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.748+0000 7f36ebe00640 1 -- 172.21.15.143:0/367846978 wait complete. 2024-01-16T15:02:10.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.750+0000 7f36ebe00640 1 Processor -- start 2024-01-16T15:02:10.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.750+0000 7f36ebe00640 1 -- start start 2024-01-16T15:02:10.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.751+0000 7f36ebe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36cc0047c0 0x7f36ec09c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:10.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.751+0000 7f36ebe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36ec1f3b40 0x7f36ec0c8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:10.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.751+0000 7f36ebe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36ec1561a0 0x7f36ec156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:10.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.751+0000 7f36ebe00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec0c6e10 con 0x7f36ec1561a0 2024-01-16T15:02:10.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.751+0000 7f36ebe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec0c7230 con 0x7f36cc0047c0 2024-01-16T15:02:10.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.751+0000 7f36ebe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec156b90 con 0x7f36ec1f3b40 2024-01-16T15:02:10.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.751+0000 7f36eb400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36ec1561a0 0x7f36ec156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:10.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.751+0000 7f36eb400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36ec1561a0 0x7f36ec156650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60926/0 (socket says 172.21.15.143:60926) 2024-01-16T15:02:10.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.751+0000 7f36eb400640 1 -- 172.21.15.143:0/1560593026 learned_addr learned my addr 172.21.15.143:0/1560593026 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:02:10.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.751+0000 7f36eaa00640 1 --2- 172.21.15.143:0/1560593026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36cc0047c0 0x7f36ec09c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:10.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.751+0000 7f36e3e00640 1 --2- 172.21.15.143:0/1560593026 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36ec1f3b40 0x7f36ec0c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:10.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.751+0000 7f36eb400640 1 -- 172.21.15.143:0/1560593026 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36ec1f3b40 msgr2=0x7f36ec0c8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:10.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.751+0000 7f36eb400640 1 --2- 172.21.15.143:0/1560593026 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36ec1f3b40 0x7f36ec0c8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:10.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.751+0000 7f36eb400640 1 -- 172.21.15.143:0/1560593026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36cc0047c0 msgr2=0x7f36ec09c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:10.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.751+0000 7f36eb400640 1 --2- 172.21.15.143:0/1560593026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36cc0047c0 0x7f36ec09c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:10.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.752+0000 7f36eb400640 1 -- 172.21.15.143:0/1560593026 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ec0c6190 con 0x7f36ec1561a0 2024-01-16T15:02:10.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.752+0000 7f36eaa00640 1 --2- 172.21.15.143:0/1560593026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36cc0047c0 0x7f36ec09c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:02:10.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.752+0000 7f36eb400640 1 --2- 172.21.15.143:0/1560593026 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36ec1561a0 0x7f36ec156650 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f36ec0c9c80 tx=0x7f36e400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:10.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.752+0000 7f36e2a00640 1 -- 172.21.15.143:0/1560593026 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e4003b70 con 0x7f36ec1561a0 2024-01-16T15:02:10.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.752+0000 7f36e2a00640 1 -- 172.21.15.143:0/1560593026 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e4003cf0 con 0x7f36ec1561a0 2024-01-16T15:02:10.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.752+0000 7f36e2a00640 1 -- 172.21.15.143:0/1560593026 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e4003ea0 con 0x7f36ec1561a0 2024-01-16T15:02:10.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.752+0000 7f36ebe00640 1 -- 172.21.15.143:0/1560593026 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36ec0c6470 con 0x7f36ec1561a0 2024-01-16T15:02:10.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.752+0000 7f36ebe00640 1 -- 172.21.15.143:0/1560593026 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36ec0c6a30 con 0x7f36ec1561a0 2024-01-16T15:02:10.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.753+0000 7f36ebe00640 1 -- 172.21.15.143:0/1560593026 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36ec0c6f50 con 0x7f36ec1561a0 2024-01-16T15:02:10.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.754+0000 7f36e2a00640 1 -- 172.21.15.143:0/1560593026 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7f36e40052b0 con 0x7f36ec1561a0 2024-01-16T15:02:10.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.755+0000 7f36e2a00640 1 --2- 172.21.15.143:0/1560593026 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f36c406b460 0x7f36c406d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:10.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.755+0000 7f36e2a00640 1 -- 172.21.15.143:0/1560593026 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f36e4089e40 con 0x7f36ec1561a0 2024-01-16T15:02:10.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.755+0000 7f36eaa00640 1 --2- 172.21.15.143:0/1560593026 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f36c406b460 0x7f36c406d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:10.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.756+0000 7f36eaa00640 1 --2- 172.21.15.143:0/1560593026 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f36c406b460 0x7f36c406d920 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f36ec064320 tx=0x7f36dc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:10.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.760+0000 7f36e2a00640 1 -- 172.21.15.143:0/1560593026 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f36ec0c6f50 con 0x7f36ec1561a0 2024-01-16T15:02:10.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.973+0000 7f36ebe00640 1 -- 172.21.15.143:0/1560593026 --> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f36ec06d5e0 con 0x7f36c406b460 2024-01-16T15:02:10.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.981+0000 7f36e2a00640 1 -- 172.21.15.143:0/1560593026 <== mgr.4113 v2:172.21.15.143:6832/3153492115 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7f36ec06d5e0 con 0x7f36c406b460 2024-01-16T15:02:10.961 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-16T15:02:10.961 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-16T15:02:10.961 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-16T15:02:10.961 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-16T15:02:10.962 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-16T15:02:10.962 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-16T15:02:10.962 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-16T15:02:10.962 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:02:10.962 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-16T15:02:10.962 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 33.33333333333333, 2024-01-16T15:02:10.962 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "1 GB/3 GB", 2024-01-16T15:02:10.962 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "1/3", 2024-01-16T15:02:10.962 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-16T15:02:10.962 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-16T15:02:10.962 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:02:10.962 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:02:10.962 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-16T15:02:10.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.983+0000 7f36ebe00640 1 -- 172.21.15.143:0/1560593026 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f36c406b460 msgr2=0x7f36c406d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:10.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.983+0000 7f36ebe00640 1 --2- 172.21.15.143:0/1560593026 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f36c406b460 0x7f36c406d920 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f36ec064320 tx=0x7f36dc008040 comp rx=0 tx=0).stop 2024-01-16T15:02:10.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.983+0000 7f36ebe00640 1 -- 172.21.15.143:0/1560593026 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36ec1561a0 msgr2=0x7f36ec156650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:10.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.983+0000 7f36ebe00640 1 --2- 172.21.15.143:0/1560593026 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36ec1561a0 0x7f36ec156650 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f36ec0c9c80 tx=0x7f36e400dc00 comp rx=0 tx=0).stop 2024-01-16T15:02:10.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.983+0000 7f36ebe00640 1 -- 172.21.15.143:0/1560593026 shutdown_connections 2024-01-16T15:02:10.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.983+0000 7f36ebe00640 1 --2- 172.21.15.143:0/1560593026 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f36c406b460 0x7f36c406d920 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:10.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.983+0000 7f36ebe00640 1 --2- 172.21.15.143:0/1560593026 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36ec1561a0 0x7f36ec156650 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:10.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.983+0000 7f36ebe00640 1 --2- 172.21.15.143:0/1560593026 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36ec1f3b40 0x7f36ec0c8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:10.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.983+0000 7f36ebe00640 1 --2- 172.21.15.143:0/1560593026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36cc0047c0 0x7f36ec09c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:10.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.984+0000 7f36ebe00640 1 -- 172.21.15.143:0/1560593026 >> 172.21.15.143:0/1560593026 conn(0x7f36ec0cbb40 msgr2=0x7f36ec0cd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:10.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.984+0000 7f36ebe00640 1 -- 172.21.15.143:0/1560593026 shutdown_connections 2024-01-16T15:02:10.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:10.984+0000 7f36ebe00640 1 -- 172.21.15.143:0/1560593026 wait complete. 2024-01-16T15:02:10.982 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-16T15:02:11.984 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-16T15:02:12.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.265+0000 7fc94a000640 1 Processor -- start 2024-01-16T15:02:12.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.266+0000 7fc94a000640 1 -- start start 2024-01-16T15:02:12.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.268+0000 7fc94a000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc9440c59b0 0x7fc944166230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:12.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.268+0000 7fc94a000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9440c5dc0 con 0x7fc9440cc0f0 2024-01-16T15:02:12.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.268+0000 7fc94a000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc94415eea0 con 0x7fc94415b440 2024-01-16T15:02:12.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.268+0000 7fc94a000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc94415b820 con 0x7fc9440c59b0 2024-01-16T15:02:12.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.269+0000 7fc942a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc9440cc0f0 0x7fc9440c52a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:38242/0 (socket says 172.21.15.143:38242) 2024-01-16T15:02:12.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.269+0000 7fc942a00640 1 -- 172.21.15.143:0/3467861312 learned_addr learned my addr 172.21.15.143:0/3467861312 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:02:12.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.270+0000 7fc943e00640 1 --2- 172.21.15.143:0/3467861312 >> v2:172.21.15.143:3300/0 conn(0x7fc9440c59b0 0x7fc944166230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:12.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.271+0000 7fc943e00640 1 -- 172.21.15.143:0/3467861312 >> v1:172.21.15.204:6789/0 conn(0x7fc94415b440 legacy=0x7fc9440cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:12.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.271+0000 7fc943e00640 1 -- 172.21.15.143:0/3467861312 >> v1:172.21.15.143:6789/0 conn(0x7fc9440cc0f0 legacy=0x7fc9440c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:12.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.271+0000 7fc943e00640 1 -- 172.21.15.143:0/3467861312 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc94406d600 con 0x7fc9440c59b0 2024-01-16T15:02:12.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.273+0000 7fc943e00640 1 --2- 172.21.15.143:0/3467861312 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9440c59b0 0x7fc944166230 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc934009a00 tx=0x7fc93402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=742664ae38a02554 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:12.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.274+0000 7fc942000640 1 -- 172.21.15.143:0/3467861312 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc934031430 con 0x7fc9440c59b0 2024-01-16T15:02:12.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.274+0000 7fc942000640 1 -- 172.21.15.143:0/3467861312 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc934031590 con 0x7fc9440c59b0 2024-01-16T15:02:12.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.274+0000 7fc942000640 1 -- 172.21.15.143:0/3467861312 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc934031920 con 0x7fc9440c59b0 2024-01-16T15:02:12.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.275+0000 7fc94a000640 1 -- 172.21.15.143:0/3467861312 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9440c59b0 msgr2=0x7fc944166230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:12.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.275+0000 7fc94a000640 1 --2- 172.21.15.143:0/3467861312 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9440c59b0 0x7fc944166230 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc934009a00 tx=0x7fc93402b960 comp rx=0 tx=0).stop 2024-01-16T15:02:12.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.275+0000 7fc94a000640 1 -- 172.21.15.143:0/3467861312 shutdown_connections 2024-01-16T15:02:12.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.275+0000 7fc94a000640 1 --2- 172.21.15.143:0/3467861312 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9440c59b0 0x7fc944166230 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:12.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.275+0000 7fc94a000640 1 -- 172.21.15.143:0/3467861312 >> 172.21.15.143:0/3467861312 conn(0x7fc9440b0c50 msgr2=0x7fc9440b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:12.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.276+0000 7fc94a000640 1 -- 172.21.15.143:0/3467861312 shutdown_connections 2024-01-16T15:02:12.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.276+0000 7fc94a000640 1 -- 172.21.15.143:0/3467861312 wait complete. 2024-01-16T15:02:12.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.279+0000 7fc94a000640 1 Processor -- start 2024-01-16T15:02:12.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.279+0000 7fc94a000640 1 -- start start 2024-01-16T15:02:12.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.281+0000 7fc94a000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc9440c59b0 0x7fc94415a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:12.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.281+0000 7fc94a000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9440cc0f0 0x7fc9440ae970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:12.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.281+0000 7fc94a000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc94415b440 0x7fc9441563a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:12.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.281+0000 7fc94a000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9440afd40 con 0x7fc94415b440 2024-01-16T15:02:12.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.281+0000 7fc94a000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9440aeeb0 con 0x7fc9440cc0f0 2024-01-16T15:02:12.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.281+0000 7fc94a000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9440af130 con 0x7fc9440c59b0 2024-01-16T15:02:12.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.282+0000 7fc943e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc94415b440 0x7fc9441563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:12.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.282+0000 7fc943e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc94415b440 0x7fc9441563a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60938/0 (socket says 172.21.15.143:60938) 2024-01-16T15:02:12.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.282+0000 7fc943e00640 1 -- 172.21.15.143:0/2844296845 learned_addr learned my addr 172.21.15.143:0/2844296845 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:02:12.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.283+0000 7fc942a00640 1 --2- 172.21.15.143:0/2844296845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9440cc0f0 0x7fc9440ae970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:12.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.283+0000 7fc943400640 1 --2- 172.21.15.143:0/2844296845 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc9440c59b0 0x7fc94415a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:12.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.283+0000 7fc943e00640 1 -- 172.21.15.143:0/2844296845 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc9440c59b0 msgr2=0x7fc94415a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:12.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.283+0000 7fc943e00640 1 --2- 172.21.15.143:0/2844296845 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc9440c59b0 0x7fc94415a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:12.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.283+0000 7fc943e00640 1 -- 172.21.15.143:0/2844296845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9440cc0f0 msgr2=0x7fc9440ae970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:12.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.283+0000 7fc943e00640 1 --2- 172.21.15.143:0/2844296845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9440cc0f0 0x7fc9440ae970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:12.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.283+0000 7fc943e00640 1 -- 172.21.15.143:0/2844296845 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc934009400 con 0x7fc94415b440 2024-01-16T15:02:12.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.284+0000 7fc943e00640 1 --2- 172.21.15.143:0/2844296845 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc94415b440 0x7fc9441563a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc934009b20 tx=0x7fc9340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:12.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.285+0000 7fc93b400640 1 -- 172.21.15.143:0/2844296845 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93403bc10 con 0x7fc94415b440 2024-01-16T15:02:12.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.285+0000 7fc93b400640 1 -- 172.21.15.143:0/2844296845 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93403bd70 con 0x7fc94415b440 2024-01-16T15:02:12.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.285+0000 7fc93b400640 1 -- 172.21.15.143:0/2844296845 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc934046690 con 0x7fc94415b440 2024-01-16T15:02:12.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.285+0000 7fc94a000640 1 -- 172.21.15.143:0/2844296845 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9440af410 con 0x7fc94415b440 2024-01-16T15:02:12.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.286+0000 7fc94a000640 1 -- 172.21.15.143:0/2844296845 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9440af9d0 con 0x7fc94415b440 2024-01-16T15:02:12.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.289+0000 7fc94a000640 1 -- 172.21.15.143:0/2844296845 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9440afef0 con 0x7fc94415b440 2024-01-16T15:02:12.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.289+0000 7fc93b400640 1 -- 172.21.15.143:0/2844296845 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7fc9340467f0 con 0x7fc94415b440 2024-01-16T15:02:12.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.290+0000 7fc93b400640 1 --2- 172.21.15.143:0/2844296845 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fc91c06b470 0x7fc91c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:12.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.291+0000 7fc93b400640 1 -- 172.21.15.143:0/2844296845 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc9340ac360 con 0x7fc94415b440 2024-01-16T15:02:12.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.291+0000 7fc943400640 1 --2- 172.21.15.143:0/2844296845 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fc91c06b470 0x7fc91c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:12.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.292+0000 7fc943400640 1 --2- 172.21.15.143:0/2844296845 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fc91c06b470 0x7fc91c06d930 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc92c001500 tx=0x7fc92c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:12.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.296+0000 7fc93b400640 1 -- 172.21.15.143:0/2844296845 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc9440afef0 con 0x7fc94415b440 2024-01-16T15:02:12.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.648+0000 7fc94a000640 1 -- 172.21.15.143:0/2844296845 --> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc94415f3d0 con 0x7fc91c06b470 2024-01-16T15:02:12.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.655+0000 7fc93b400640 1 -- 172.21.15.143:0/2844296845 <== mgr.4113 v2:172.21.15.143:6832/3153492115 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7fc92c0011e0 con 0x7fc91c06b470 2024-01-16T15:02:12.636 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-16T15:02:12.636 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-16T15:02:12.636 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-16T15:02:12.636 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-16T15:02:12.636 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-16T15:02:12.636 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-16T15:02:12.636 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-16T15:02:12.636 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:02:12.637 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-16T15:02:12.637 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 33.33333333333333, 2024-01-16T15:02:12.637 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "1 GB/3 GB", 2024-01-16T15:02:12.637 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "1/3", 2024-01-16T15:02:12.637 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-16T15:02:12.637 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-16T15:02:12.637 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:02:12.637 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:02:12.637 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-16T15:02:12.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.659+0000 7fc94a000640 1 -- 172.21.15.143:0/2844296845 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fc91c06b470 msgr2=0x7fc91c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:12.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.659+0000 7fc94a000640 1 --2- 172.21.15.143:0/2844296845 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fc91c06b470 0x7fc91c06d930 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc92c001500 tx=0x7fc92c009040 comp rx=0 tx=0).stop 2024-01-16T15:02:12.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.659+0000 7fc94a000640 1 -- 172.21.15.143:0/2844296845 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc94415b440 msgr2=0x7fc9441563a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:12.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.659+0000 7fc94a000640 1 --2- 172.21.15.143:0/2844296845 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc94415b440 0x7fc9441563a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc934009b20 tx=0x7fc9340046c0 comp rx=0 tx=0).stop 2024-01-16T15:02:12.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.659+0000 7fc94a000640 1 -- 172.21.15.143:0/2844296845 shutdown_connections 2024-01-16T15:02:12.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.659+0000 7fc94a000640 1 --2- 172.21.15.143:0/2844296845 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fc91c06b470 0x7fc91c06d930 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:12.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.659+0000 7fc94a000640 1 --2- 172.21.15.143:0/2844296845 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc94415b440 0x7fc9441563a0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:12.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.659+0000 7fc94a000640 1 --2- 172.21.15.143:0/2844296845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9440cc0f0 0x7fc9440ae970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:12.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.659+0000 7fc94a000640 1 --2- 172.21.15.143:0/2844296845 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc9440c59b0 0x7fc94415a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:12.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.659+0000 7fc94a000640 1 -- 172.21.15.143:0/2844296845 >> 172.21.15.143:0/2844296845 conn(0x7fc9440b0c50 msgr2=0x7fc944071170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:12.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.660+0000 7fc94a000640 1 -- 172.21.15.143:0/2844296845 shutdown_connections 2024-01-16T15:02:12.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:12.660+0000 7fc94a000640 1 -- 172.21.15.143:0/2844296845 wait complete. 2024-01-16T15:02:12.667 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-16T15:02:13.668 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-16T15:02:14.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.029+0000 7f2d34e00640 1 Processor -- start 2024-01-16T15:02:14.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.030+0000 7f2d34e00640 1 -- start start 2024-01-16T15:02:14.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.031+0000 7f2d34e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d301660b0 0x7f2d30162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:14.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.031+0000 7f2d34e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d30166510 con 0x7f2d30162540 2024-01-16T15:02:14.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.031+0000 7f2d34e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d301655f0 con 0x7f2d300ccbd0 2024-01-16T15:02:14.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.031+0000 7f2d34e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d300cc380 con 0x7f2d301660b0 2024-01-16T15:02:14.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.032+0000 7f2d2e800640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f2d30162540 0x7f2d300c9650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:38246/0 (socket says 172.21.15.143:38246) 2024-01-16T15:02:14.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.032+0000 7f2d2e800640 1 -- 172.21.15.143:0/2833761445 learned_addr learned my addr 172.21.15.143:0/2833761445 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:02:14.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.032+0000 7f2d27e00640 1 --2- 172.21.15.143:0/2833761445 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d301660b0 0x7f2d30162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:14.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.033+0000 7f2d2d400640 1 -- 172.21.15.143:0/2833761445 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 858486113 0 0) 0x7f2d301655f0 con 0x7f2d300ccbd0 2024-01-16T15:02:14.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.033+0000 7f2d2d400640 1 -- 172.21.15.143:0/2833761445 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d10003520 con 0x7f2d300ccbd0 2024-01-16T15:02:14.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.033+0000 7f2d2d400640 1 -- 172.21.15.143:0/2833761445 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4172717290 0 0) 0x7f2d28001490 con 0x7f2d30162540 2024-01-16T15:02:14.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.033+0000 7f2d2d400640 1 -- 172.21.15.143:0/2833761445 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d301655f0 con 0x7f2d30162540 2024-01-16T15:02:14.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.034+0000 7f2d2d400640 1 -- 172.21.15.143:0/2833761445 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3921216461 0 0) 0x7f2d10003520 con 0x7f2d300ccbd0 2024-01-16T15:02:14.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.034+0000 7f2d27e00640 1 -- 172.21.15.143:0/2833761445 >> v1:172.21.15.204:6789/0 conn(0x7f2d300ccbd0 legacy=0x7f2d300ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:14.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.034+0000 7f2d27e00640 1 -- 172.21.15.143:0/2833761445 >> v1:172.21.15.143:6789/0 conn(0x7f2d30162540 legacy=0x7f2d300c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:14.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.034+0000 7f2d27e00640 1 -- 172.21.15.143:0/2833761445 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d30162b30 con 0x7f2d301660b0 2024-01-16T15:02:14.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.035+0000 7f2d27e00640 1 --2- 172.21.15.143:0/2833761445 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d301660b0 0x7f2d30162000 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2d301657e0 tx=0x7f2d1802b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b56b5f1262b2ba21 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:14.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.036+0000 7f2d2d400640 1 -- 172.21.15.143:0/2833761445 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d28001b00 con 0x7f2d301660b0 2024-01-16T15:02:14.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.036+0000 7f2d2d400640 1 -- 172.21.15.143:0/2833761445 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d1802e050 con 0x7f2d301660b0 2024-01-16T15:02:14.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.036+0000 7f2d2d400640 1 -- 172.21.15.143:0/2833761445 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d20001670 con 0x7f2d301660b0 2024-01-16T15:02:14.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.038+0000 7f2d34e00640 1 -- 172.21.15.143:0/2833761445 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d301660b0 msgr2=0x7f2d30162000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:14.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.038+0000 7f2d34e00640 1 --2- 172.21.15.143:0/2833761445 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d301660b0 0x7f2d30162000 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2d301657e0 tx=0x7f2d1802b710 comp rx=0 tx=0).stop 2024-01-16T15:02:14.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.042+0000 7f2d34e00640 1 -- 172.21.15.143:0/2833761445 shutdown_connections 2024-01-16T15:02:14.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.042+0000 7f2d34e00640 1 --2- 172.21.15.143:0/2833761445 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d301660b0 0x7f2d30162000 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:14.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.042+0000 7f2d34e00640 1 -- 172.21.15.143:0/2833761445 >> 172.21.15.143:0/2833761445 conn(0x7f2d300b0c50 msgr2=0x7f2d300b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:14.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.043+0000 7f2d34e00640 1 -- 172.21.15.143:0/2833761445 shutdown_connections 2024-01-16T15:02:14.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.043+0000 7f2d34e00640 1 -- 172.21.15.143:0/2833761445 wait complete. 2024-01-16T15:02:14.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.047+0000 7f2d34e00640 1 Processor -- start 2024-01-16T15:02:14.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.048+0000 7f2d34e00640 1 -- start start 2024-01-16T15:02:14.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.049+0000 7f2d34e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d300ccbd0 0x7f2d301556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:14.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.049+0000 7f2d34e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d30162540 0x7f2d30071e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:14.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.049+0000 7f2d34e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d301660b0 0x7f2d300723b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:14.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.049+0000 7f2d34e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d300710d0 con 0x7f2d30162540 2024-01-16T15:02:14.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.049+0000 7f2d34e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d300704c0 con 0x7f2d300ccbd0 2024-01-16T15:02:14.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.049+0000 7f2d34e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d300707a0 con 0x7f2d301660b0 2024-01-16T15:02:14.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.050+0000 7f2d27e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d30162540 0x7f2d30071e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:14.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.050+0000 7f2d27e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d30162540 0x7f2d30071e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60942/0 (socket says 172.21.15.143:60942) 2024-01-16T15:02:14.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.051+0000 7f2d27e00640 1 -- 172.21.15.143:0/1252847925 learned_addr learned my addr 172.21.15.143:0/1252847925 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:02:14.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.050+0000 7f2d2de00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d300ccbd0 0x7f2d301556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:14.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.050+0000 7f2d2e800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d301660b0 0x7f2d300723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:14.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.052+0000 7f2d27e00640 1 -- 172.21.15.143:0/1252847925 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d301660b0 msgr2=0x7f2d300723b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:14.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.052+0000 7f2d27e00640 1 --2- 172.21.15.143:0/1252847925 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d301660b0 0x7f2d300723b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:14.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.052+0000 7f2d27e00640 1 -- 172.21.15.143:0/1252847925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d300ccbd0 msgr2=0x7f2d301556b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:14.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.052+0000 7f2d27e00640 1 --2- 172.21.15.143:0/1252847925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d300ccbd0 0x7f2d301556b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:14.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.052+0000 7f2d27e00640 1 -- 172.21.15.143:0/1252847925 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d18009400 con 0x7f2d30162540 2024-01-16T15:02:14.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.054+0000 7f2d27e00640 1 --2- 172.21.15.143:0/1252847925 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d30162540 0x7f2d30071e70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2d180098f0 tx=0x7f2d180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:14.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.054+0000 7f2d26000640 1 -- 172.21.15.143:0/1252847925 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d180327c0 con 0x7f2d30162540 2024-01-16T15:02:14.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.054+0000 7f2d26000640 1 -- 172.21.15.143:0/1252847925 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d18032920 con 0x7f2d30162540 2024-01-16T15:02:14.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.054+0000 7f2d26000640 1 -- 172.21.15.143:0/1252847925 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d18032cb0 con 0x7f2d30162540 2024-01-16T15:02:14.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.054+0000 7f2d34e00640 1 -- 172.21.15.143:0/1252847925 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d30070a80 con 0x7f2d30162540 2024-01-16T15:02:14.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.055+0000 7f2d34e00640 1 -- 172.21.15.143:0/1252847925 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d30071560 con 0x7f2d30162540 2024-01-16T15:02:14.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.057+0000 7f2d34e00640 1 -- 172.21.15.143:0/1252847925 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d300716f0 con 0x7f2d30162540 2024-01-16T15:02:14.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.058+0000 7f2d26000640 1 -- 172.21.15.143:0/1252847925 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7f2d18039070 con 0x7f2d30162540 2024-01-16T15:02:14.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.060+0000 7f2d26000640 1 --2- 172.21.15.143:0/1252847925 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f2d0806b4c0 0x7f2d0806d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:14.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.060+0000 7f2d26000640 1 -- 172.21.15.143:0/1252847925 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d180ac090 con 0x7f2d30162540 2024-01-16T15:02:14.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.065+0000 7f2d2de00640 1 --2- 172.21.15.143:0/1252847925 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f2d0806b4c0 0x7f2d0806d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:14.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.066+0000 7f2d2de00640 1 --2- 172.21.15.143:0/1252847925 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f2d0806b4c0 0x7f2d0806d980 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2d20001380 tx=0x7f2d20009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:14.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.067+0000 7f2d26000640 1 -- 172.21.15.143:0/1252847925 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2d18076c50 con 0x7f2d30162540 2024-01-16T15:02:14.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.318+0000 7f2d34e00640 1 -- 172.21.15.143:0/1252847925 --> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d30165b90 con 0x7f2d0806b4c0 2024-01-16T15:02:14.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.329+0000 7f2d26000640 1 -- 172.21.15.143:0/1252847925 <== mgr.4113 v2:172.21.15.143:6832/3153492115 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7f2d20001c50 con 0x7f2d0806b4c0 2024-01-16T15:02:14.310 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-16T15:02:14.310 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-16T15:02:14.310 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-16T15:02:14.310 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-16T15:02:14.310 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-16T15:02:14.310 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-16T15:02:14.310 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-16T15:02:14.310 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:02:14.310 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-16T15:02:14.311 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 66.66666666666666, 2024-01-16T15:02:14.311 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "2 GB/3 GB", 2024-01-16T15:02:14.311 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "2/3", 2024-01-16T15:02:14.311 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-16T15:02:14.311 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-16T15:02:14.311 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:02:14.311 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:02:14.311 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-16T15:02:14.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.332+0000 7f2d34e00640 1 -- 172.21.15.143:0/1252847925 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f2d0806b4c0 msgr2=0x7f2d0806d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:14.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.332+0000 7f2d34e00640 1 --2- 172.21.15.143:0/1252847925 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f2d0806b4c0 0x7f2d0806d980 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2d20001380 tx=0x7f2d20009040 comp rx=0 tx=0).stop 2024-01-16T15:02:14.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.332+0000 7f2d34e00640 1 -- 172.21.15.143:0/1252847925 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d30162540 msgr2=0x7f2d30071e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:14.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.332+0000 7f2d34e00640 1 --2- 172.21.15.143:0/1252847925 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d30162540 0x7f2d30071e70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2d180098f0 tx=0x7f2d180046c0 comp rx=0 tx=0).stop 2024-01-16T15:02:14.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.332+0000 7f2d34e00640 1 -- 172.21.15.143:0/1252847925 shutdown_connections 2024-01-16T15:02:14.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.332+0000 7f2d34e00640 1 --2- 172.21.15.143:0/1252847925 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f2d0806b4c0 0x7f2d0806d980 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:14.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.332+0000 7f2d34e00640 1 --2- 172.21.15.143:0/1252847925 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d301660b0 0x7f2d300723b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:14.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.332+0000 7f2d34e00640 1 --2- 172.21.15.143:0/1252847925 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d30162540 0x7f2d30071e70 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:14.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.332+0000 7f2d34e00640 1 --2- 172.21.15.143:0/1252847925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d300ccbd0 0x7f2d301556b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:14.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.332+0000 7f2d34e00640 1 -- 172.21.15.143:0/1252847925 >> 172.21.15.143:0/1252847925 conn(0x7f2d300b0c50 msgr2=0x7f2d3006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:14.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.332+0000 7f2d34e00640 1 -- 172.21.15.143:0/1252847925 shutdown_connections 2024-01-16T15:02:14.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:14.332+0000 7f2d34e00640 1 -- 172.21.15.143:0/1252847925 wait complete. 2024-01-16T15:02:14.328 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-16T15:02:15.330 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-16T15:02:15.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.569+0000 7fb6eec00640 1 Processor -- start 2024-01-16T15:02:15.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.569+0000 7fb6eec00640 1 -- start start 2024-01-16T15:02:15.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.571+0000 7fb6eec00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb6e815b440 0x7fb6e80cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:15.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.571+0000 7fb6eec00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb6e80cbf20 0x7fb6e80c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:15.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.571+0000 7fb6eec00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb6e80c57e0 0x7fb6e80c3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:15.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.571+0000 7fb6eec00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6e80cc3d0 con 0x7fb6e815b440 2024-01-16T15:02:15.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.571+0000 7fb6eec00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6e815b820 con 0x7fb6e80cbf20 2024-01-16T15:02:15.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.571+0000 7fb6eec00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e815eea0 con 0x7fb6e80c57e0 2024-01-16T15:02:15.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.571+0000 7fb6e7e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb6e815b440 0x7fb6e80cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:15.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.571+0000 7fb6e7e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb6e815b440 0x7fb6e80cb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34276/0 (socket says 172.21.15.143:34276) 2024-01-16T15:02:15.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.571+0000 7fb6e7e00640 1 -- 172.21.15.143:0/3487594183 learned_addr learned my addr 172.21.15.143:0/3487594183 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:02:15.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.571+0000 7fb6dfe00640 1 --2- 172.21.15.143:0/3487594183 >> v2:172.21.15.204:3300/0 conn(0x7fb6e80cbf20 0x7fb6e80c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:15.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.571+0000 7fb6ecc00640 1 --2- 172.21.15.143:0/3487594183 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb6e80c57e0 0x7fb6e80c3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:15.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.572+0000 7fb6e7e00640 1 -- 172.21.15.143:0/3487594183 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb6e80c57e0 msgr2=0x7fb6e80c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:15.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.572+0000 7fb6e7e00640 1 --2- 172.21.15.143:0/3487594183 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb6e80c57e0 0x7fb6e80c3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:15.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.572+0000 7fb6e7e00640 1 -- 172.21.15.143:0/3487594183 >> v2:172.21.15.204:3300/0 conn(0x7fb6e80cbf20 msgr2=0x7fb6e80c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:15.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.572+0000 7fb6e7e00640 1 --2- 172.21.15.143:0/3487594183 >> v2:172.21.15.204:3300/0 conn(0x7fb6e80cbf20 0x7fb6e80c52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:15.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.572+0000 7fb6e7e00640 1 -- 172.21.15.143:0/3487594183 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e80abf30 con 0x7fb6e815b440 2024-01-16T15:02:15.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.573+0000 7fb6e7e00640 1 --2- 172.21.15.143:0/3487594183 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6e815b440 0x7fb6e80cb9e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb6e806e890 tx=0x7fb6d802b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769a47223b00f960 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:15.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.573+0000 7fb6e7400640 1 -- 172.21.15.143:0/3487594183 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d80317d0 con 0x7fb6e815b440 2024-01-16T15:02:15.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.573+0000 7fb6e7400640 1 -- 172.21.15.143:0/3487594183 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d8031930 con 0x7fb6e815b440 2024-01-16T15:02:15.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.573+0000 7fb6e7400640 1 -- 172.21.15.143:0/3487594183 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8031cc0 con 0x7fb6e815b440 2024-01-16T15:02:15.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.574+0000 7fb6eec00640 1 -- 172.21.15.143:0/3487594183 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6e815b440 msgr2=0x7fb6e80cb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:15.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.574+0000 7fb6eec00640 1 --2- 172.21.15.143:0/3487594183 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6e815b440 0x7fb6e80cb9e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb6e806e890 tx=0x7fb6d802b870 comp rx=0 tx=0).stop 2024-01-16T15:02:15.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.574+0000 7fb6eec00640 1 -- 172.21.15.143:0/3487594183 shutdown_connections 2024-01-16T15:02:15.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.574+0000 7fb6eec00640 1 --2- 172.21.15.143:0/3487594183 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb6e80c57e0 0x7fb6e80c3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:15.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.574+0000 7fb6eec00640 1 --2- 172.21.15.143:0/3487594183 >> v2:172.21.15.204:3300/0 conn(0x7fb6e80cbf20 0x7fb6e80c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:15.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.574+0000 7fb6eec00640 1 --2- 172.21.15.143:0/3487594183 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6e815b440 0x7fb6e80cb9e0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:15.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.574+0000 7fb6eec00640 1 -- 172.21.15.143:0/3487594183 >> 172.21.15.143:0/3487594183 conn(0x7fb6e80b0c50 msgr2=0x7fb6e80b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:15.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.574+0000 7fb6eec00640 1 -- 172.21.15.143:0/3487594183 shutdown_connections 2024-01-16T15:02:15.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.574+0000 7fb6eec00640 1 -- 172.21.15.143:0/3487594183 wait complete. 2024-01-16T15:02:15.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.576+0000 7fb6eec00640 1 Processor -- start 2024-01-16T15:02:15.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.576+0000 7fb6eec00640 1 -- start start 2024-01-16T15:02:15.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.577+0000 7fb6eec00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6e80c57e0 0x7fb6e80c6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:15.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.577+0000 7fb6eec00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6e80cbf20 0x7fb6e815dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:15.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.577+0000 7fb6eec00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb6e815b440 0x7fb6e80b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:15.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.577+0000 7fb6eec00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e80c81d0 con 0x7fb6e80cbf20 2024-01-16T15:02:15.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.577+0000 7fb6eec00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e815e160 con 0x7fb6e80c57e0 2024-01-16T15:02:15.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.577+0000 7fb6eec00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e80c7130 con 0x7fb6e815b440 2024-01-16T15:02:15.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.577+0000 7fb6dfe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6e80cbf20 0x7fb6e815dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:15.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.577+0000 7fb6dfe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6e80cbf20 0x7fb6e815dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34288/0 (socket says 172.21.15.143:34288) 2024-01-16T15:02:15.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.577+0000 7fb6dfe00640 1 -- 172.21.15.143:0/2062856480 learned_addr learned my addr 172.21.15.143:0/2062856480 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:02:15.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.577+0000 7fb6e7e00640 1 --2- 172.21.15.143:0/2062856480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6e80c57e0 0x7fb6e80c6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:15.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.577+0000 7fb6ecc00640 1 --2- 172.21.15.143:0/2062856480 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb6e815b440 0x7fb6e80b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:15.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.577+0000 7fb6dfe00640 1 -- 172.21.15.143:0/2062856480 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb6e815b440 msgr2=0x7fb6e80b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:15.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.577+0000 7fb6dfe00640 1 --2- 172.21.15.143:0/2062856480 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb6e815b440 0x7fb6e80b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:15.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.577+0000 7fb6dfe00640 1 -- 172.21.15.143:0/2062856480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6e80c57e0 msgr2=0x7fb6e80c6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:15.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.577+0000 7fb6dfe00640 1 --2- 172.21.15.143:0/2062856480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6e80c57e0 0x7fb6e80c6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:15.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.577+0000 7fb6dfe00640 1 -- 172.21.15.143:0/2062856480 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d80094d0 con 0x7fb6e80cbf20 2024-01-16T15:02:15.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.577+0000 7fb6ecc00640 1 --2- 172.21.15.143:0/2062856480 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb6e815b440 0x7fb6e80b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:02:15.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.577+0000 7fb6e7e00640 1 --2- 172.21.15.143:0/2062856480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6e80c57e0 0x7fb6e80c6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:02:15.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.578+0000 7fb6dfe00640 1 --2- 172.21.15.143:0/2062856480 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6e80cbf20 0x7fb6e815dc20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb6d000d800 tx=0x7fb6d000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:15.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.578+0000 7fb6e4c00640 1 -- 172.21.15.143:0/2062856480 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d000b6c0 con 0x7fb6e80cbf20 2024-01-16T15:02:15.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.578+0000 7fb6e4c00640 1 -- 172.21.15.143:0/2062856480 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d000b840 con 0x7fb6e80cbf20 2024-01-16T15:02:15.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.578+0000 7fb6e4c00640 1 -- 172.21.15.143:0/2062856480 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d000bbf0 con 0x7fb6e80cbf20 2024-01-16T15:02:15.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.578+0000 7fb6eec00640 1 -- 172.21.15.143:0/2062856480 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e80c7410 con 0x7fb6e80cbf20 2024-01-16T15:02:15.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.578+0000 7fb6eec00640 1 -- 172.21.15.143:0/2062856480 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e80c79d0 con 0x7fb6e80cbf20 2024-01-16T15:02:15.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.579+0000 7fb6eec00640 1 -- 172.21.15.143:0/2062856480 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6e804e8b0 con 0x7fb6e80cbf20 2024-01-16T15:02:15.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.580+0000 7fb6e4c00640 1 -- 172.21.15.143:0/2062856480 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7fb6d000bd50 con 0x7fb6e80cbf20 2024-01-16T15:02:15.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.581+0000 7fb6e4c00640 1 --2- 172.21.15.143:0/2062856480 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb6c006b4a0 0x7fb6c006d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:15.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.582+0000 7fb6e4c00640 1 -- 172.21.15.143:0/2062856480 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb6d008a5a0 con 0x7fb6e80cbf20 2024-01-16T15:02:15.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.582+0000 7fb6e7e00640 1 --2- 172.21.15.143:0/2062856480 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb6c006b4a0 0x7fb6c006d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:15.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.582+0000 7fb6e7e00640 1 --2- 172.21.15.143:0/2062856480 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb6c006b4a0 0x7fb6c006d960 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8036000 tx=0x7fb6d803d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:15.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.586+0000 7fb6e4c00640 1 -- 172.21.15.143:0/2062856480 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb6d0055060 con 0x7fb6e80cbf20 2024-01-16T15:02:15.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.811+0000 7fb6eec00640 1 -- 172.21.15.143:0/2062856480 --> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7fb6e815f6b0 con 0x7fb6c006b4a0 2024-01-16T15:02:15.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.818+0000 7fb6e4c00640 1 -- 172.21.15.143:0/2062856480 <== mgr.4113 v2:172.21.15.143:6832/3153492115 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7fb6d8031aa0 con 0x7fb6c006b4a0 2024-01-16T15:02:15.798 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-16T15:02:15.799 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-16T15:02:15.799 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-16T15:02:15.799 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-16T15:02:15.799 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-16T15:02:15.799 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-16T15:02:15.799 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-16T15:02:15.799 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:02:15.799 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-16T15:02:15.799 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 66.66666666666666, 2024-01-16T15:02:15.799 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "2 GB/3 GB", 2024-01-16T15:02:15.799 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "2/3", 2024-01-16T15:02:15.800 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-16T15:02:15.800 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-16T15:02:15.800 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:02:15.800 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:02:15.800 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-16T15:02:15.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.820+0000 7fb6eec00640 1 -- 172.21.15.143:0/2062856480 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb6c006b4a0 msgr2=0x7fb6c006d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:15.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.820+0000 7fb6eec00640 1 --2- 172.21.15.143:0/2062856480 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb6c006b4a0 0x7fb6c006d960 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8036000 tx=0x7fb6d803d040 comp rx=0 tx=0).stop 2024-01-16T15:02:15.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.820+0000 7fb6eec00640 1 -- 172.21.15.143:0/2062856480 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6e80cbf20 msgr2=0x7fb6e815dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:15.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.820+0000 7fb6eec00640 1 --2- 172.21.15.143:0/2062856480 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6e80cbf20 0x7fb6e815dc20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb6d000d800 tx=0x7fb6d000dcc0 comp rx=0 tx=0).stop 2024-01-16T15:02:15.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.820+0000 7fb6eec00640 1 -- 172.21.15.143:0/2062856480 shutdown_connections 2024-01-16T15:02:15.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.820+0000 7fb6eec00640 1 --2- 172.21.15.143:0/2062856480 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fb6c006b4a0 0x7fb6c006d960 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:15.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.820+0000 7fb6eec00640 1 --2- 172.21.15.143:0/2062856480 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb6e815b440 0x7fb6e80b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:15.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.820+0000 7fb6eec00640 1 --2- 172.21.15.143:0/2062856480 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6e80cbf20 0x7fb6e815dc20 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:15.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.820+0000 7fb6eec00640 1 --2- 172.21.15.143:0/2062856480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6e80c57e0 0x7fb6e80c6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:15.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.821+0000 7fb6eec00640 1 -- 172.21.15.143:0/2062856480 >> 172.21.15.143:0/2062856480 conn(0x7fb6e80b0c50 msgr2=0x7fb6e80ad360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:15.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.821+0000 7fb6eec00640 1 -- 172.21.15.143:0/2062856480 shutdown_connections 2024-01-16T15:02:15.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:15.821+0000 7fb6eec00640 1 -- 172.21.15.143:0/2062856480 wait complete. 2024-01-16T15:02:15.819 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-16T15:02:16.820 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-16T15:02:17.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.170+0000 7f268f200640 1 Processor -- start 2024-01-16T15:02:17.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.170+0000 7f268f200640 1 -- start start 2024-01-16T15:02:17.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.172+0000 7f268f200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f26880b3650 0x7f268806ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:17.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.172+0000 7f268f200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f268806bfd0 0x7f268815e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:17.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.172+0000 7f268f200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f268806cd80 con 0x7f2688146b10 2024-01-16T15:02:17.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.172+0000 7f268f200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f268806c7c0 con 0x7f268806bfd0 2024-01-16T15:02:17.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.172+0000 7f268f200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f268806caa0 con 0x7f26880b3650 2024-01-16T15:02:17.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.172+0000 7f268e800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f268806bfd0 0x7f268815e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:17.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.172+0000 7f268e800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f268806bfd0 0x7f268815e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34292/0 (socket says 172.21.15.143:34292) 2024-01-16T15:02:17.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.172+0000 7f268e800640 1 -- 172.21.15.143:0/3798940763 learned_addr learned my addr 172.21.15.143:0/3798940763 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:02:17.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.173+0000 7f268e800640 1 -- 172.21.15.143:0/3798940763 >> v2:172.21.15.204:3300/0 conn(0x7f26880b3650 msgr2=0x7f268806ba90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-16T15:02:17.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.173+0000 7f268e800640 1 --2- 172.21.15.143:0/3798940763 >> v2:172.21.15.204:3300/0 conn(0x7f26880b3650 0x7f268806ba90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:17.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.173+0000 7f268e800640 1 -- 172.21.15.143:0/3798940763 >> v1:172.21.15.204:6789/0 conn(0x7f2688146b10 legacy=0x7f26880b2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:17.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.173+0000 7f268e800640 1 -- 172.21.15.143:0/3798940763 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f268815da30 con 0x7f268806bfd0 2024-01-16T15:02:17.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.176+0000 7f268e800640 1 --2- 172.21.15.143:0/3798940763 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f268806bfd0 0x7f268815e230 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2688061f40 tx=0x7f268002bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89393e367564378e server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:17.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.177+0000 7f268ca00640 1 -- 172.21.15.143:0/3798940763 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268002caa0 con 0x7f268806bfd0 2024-01-16T15:02:17.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.177+0000 7f268ca00640 1 -- 172.21.15.143:0/3798940763 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268002cc00 con 0x7f268806bfd0 2024-01-16T15:02:17.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.177+0000 7f268ca00640 1 -- 172.21.15.143:0/3798940763 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26800343f0 con 0x7f268806bfd0 2024-01-16T15:02:17.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.178+0000 7f268f200640 1 -- 172.21.15.143:0/3798940763 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f268806bfd0 msgr2=0x7f268815e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:17.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.178+0000 7f268f200640 1 --2- 172.21.15.143:0/3798940763 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f268806bfd0 0x7f268815e230 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2688061f40 tx=0x7f268002bf00 comp rx=0 tx=0).stop 2024-01-16T15:02:17.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.180+0000 7f268f200640 1 -- 172.21.15.143:0/3798940763 shutdown_connections 2024-01-16T15:02:17.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.180+0000 7f268f200640 1 --2- 172.21.15.143:0/3798940763 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f268806bfd0 0x7f268815e230 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:17.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.180+0000 7f268f200640 1 --2- 172.21.15.143:0/3798940763 >> v2:172.21.15.204:3300/0 conn(0x7f26880b3650 0x7f268806ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:17.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.180+0000 7f268f200640 1 -- 172.21.15.143:0/3798940763 >> 172.21.15.143:0/3798940763 conn(0x7f2688061100 msgr2=0x7f2688061510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:17.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.180+0000 7f268f200640 1 -- 172.21.15.143:0/3798940763 shutdown_connections 2024-01-16T15:02:17.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.180+0000 7f268f200640 1 -- 172.21.15.143:0/3798940763 wait complete. 2024-01-16T15:02:17.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.183+0000 7f268f200640 1 Processor -- start 2024-01-16T15:02:17.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.183+0000 7f268f200640 1 -- start start 2024-01-16T15:02:17.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.185+0000 7f268f200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f268806bfd0 0x7f26881f3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:17.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.185+0000 7f268f200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26880b3650 0x7f268813c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:17.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.185+0000 7f268f200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2688146b10 0x7f26881f0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:17.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.185+0000 7f268f200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268813d390 con 0x7f26880b3650 2024-01-16T15:02:17.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.185+0000 7f268f200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268813c8e0 con 0x7f268806bfd0 2024-01-16T15:02:17.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.185+0000 7f268f200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f268813caf0 con 0x7f2688146b10 2024-01-16T15:02:17.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.186+0000 7f268d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26880b3650 0x7f268813c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:17.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.186+0000 7f268d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26880b3650 0x7f268813c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34306/0 (socket says 172.21.15.143:34306) 2024-01-16T15:02:17.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.186+0000 7f268e800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2688146b10 0x7f26881f0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:17.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.186+0000 7f268d400640 1 -- 172.21.15.143:0/4112005601 learned_addr learned my addr 172.21.15.143:0/4112005601 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:02:17.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.186+0000 7f268de00640 1 --2- 172.21.15.143:0/4112005601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f268806bfd0 0x7f26881f3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:17.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.186+0000 7f268d400640 1 -- 172.21.15.143:0/4112005601 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2688146b10 msgr2=0x7f26881f0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:17.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.186+0000 7f268d400640 1 --2- 172.21.15.143:0/4112005601 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2688146b10 0x7f26881f0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:17.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.186+0000 7f268d400640 1 -- 172.21.15.143:0/4112005601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f268806bfd0 msgr2=0x7f26881f3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:17.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.186+0000 7f268d400640 1 --2- 172.21.15.143:0/4112005601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f268806bfd0 0x7f26881f3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:17.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.186+0000 7f268d400640 1 -- 172.21.15.143:0/4112005601 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2680009b30 con 0x7f26880b3650 2024-01-16T15:02:17.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.187+0000 7f268d400640 1 --2- 172.21.15.143:0/4112005601 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26880b3650 0x7f268813c3a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f267000b450 tx=0x7f267000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:17.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.187+0000 7f2686000640 1 -- 172.21.15.143:0/4112005601 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2670011890 con 0x7f26880b3650 2024-01-16T15:02:17.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.187+0000 7f2686000640 1 -- 172.21.15.143:0/4112005601 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2670011a10 con 0x7f26880b3650 2024-01-16T15:02:17.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.187+0000 7f2686000640 1 -- 172.21.15.143:0/4112005601 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2670011dc0 con 0x7f26880b3650 2024-01-16T15:02:17.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.187+0000 7f268f200640 1 -- 172.21.15.143:0/4112005601 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f268813cdd0 con 0x7f26880b3650 2024-01-16T15:02:17.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.187+0000 7f268f200640 1 -- 172.21.15.143:0/4112005601 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26880cad40 con 0x7f26880b3650 2024-01-16T15:02:17.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.189+0000 7f268f200640 1 -- 172.21.15.143:0/4112005601 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26880caba0 con 0x7f26880b3650 2024-01-16T15:02:17.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.190+0000 7f2686000640 1 -- 172.21.15.143:0/4112005601 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7f267001a400 con 0x7f26880b3650 2024-01-16T15:02:17.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.191+0000 7f2686000640 1 --2- 172.21.15.143:0/4112005601 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f266006b4c0 0x7f266006d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:17.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.191+0000 7f2686000640 1 -- 172.21.15.143:0/4112005601 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f267008a270 con 0x7f26880b3650 2024-01-16T15:02:17.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.191+0000 7f268de00640 1 --2- 172.21.15.143:0/4112005601 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f266006b4c0 0x7f266006d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:17.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.192+0000 7f268de00640 1 --2- 172.21.15.143:0/4112005601 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f266006b4c0 0x7f266006d980 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2678006fd0 tx=0x7f2678008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:17.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.197+0000 7f2686000640 1 -- 172.21.15.143:0/4112005601 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f26880caba0 con 0x7f26880b3650 2024-01-16T15:02:17.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.534+0000 7f268f200640 1 -- 172.21.15.143:0/4112005601 --> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2688146ef0 con 0x7f266006b4c0 2024-01-16T15:02:17.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.547+0000 7f2686000640 1 -- 172.21.15.143:0/4112005601 <== mgr.4113 v2:172.21.15.143:6832/3153492115 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7f267800bd80 con 0x7f266006b4c0 2024-01-16T15:02:17.527 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-16T15:02:17.528 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-16T15:02:17.528 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-16T15:02:17.528 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-16T15:02:17.528 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-16T15:02:17.528 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-16T15:02:17.528 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-16T15:02:17.528 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:02:17.528 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-16T15:02:17.528 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 66.66666666666666, 2024-01-16T15:02:17.528 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "2 GB/3 GB", 2024-01-16T15:02:17.528 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "2/3", 2024-01-16T15:02:17.529 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-16T15:02:17.529 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-16T15:02:17.529 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:02:17.529 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:02:17.529 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-16T15:02:17.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.550+0000 7f268f200640 1 -- 172.21.15.143:0/4112005601 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f266006b4c0 msgr2=0x7f266006d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:17.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.550+0000 7f268f200640 1 --2- 172.21.15.143:0/4112005601 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f266006b4c0 0x7f266006d980 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2678006fd0 tx=0x7f2678008040 comp rx=0 tx=0).stop 2024-01-16T15:02:17.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.550+0000 7f268f200640 1 -- 172.21.15.143:0/4112005601 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26880b3650 msgr2=0x7f268813c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:17.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.550+0000 7f268f200640 1 --2- 172.21.15.143:0/4112005601 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26880b3650 0x7f268813c3a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f267000b450 tx=0x7f267000b910 comp rx=0 tx=0).stop 2024-01-16T15:02:17.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.551+0000 7f268f200640 1 -- 172.21.15.143:0/4112005601 shutdown_connections 2024-01-16T15:02:17.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.551+0000 7f268f200640 1 --2- 172.21.15.143:0/4112005601 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f266006b4c0 0x7f266006d980 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:17.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.551+0000 7f268f200640 1 --2- 172.21.15.143:0/4112005601 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2688146b10 0x7f26881f0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:17.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.551+0000 7f268f200640 1 --2- 172.21.15.143:0/4112005601 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26880b3650 0x7f268813c3a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:17.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.551+0000 7f268f200640 1 --2- 172.21.15.143:0/4112005601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f268806bfd0 0x7f26881f3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:17.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.551+0000 7f268f200640 1 -- 172.21.15.143:0/4112005601 >> 172.21.15.143:0/4112005601 conn(0x7f2688061100 msgr2=0x7f26880734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:17.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.551+0000 7f268f200640 1 -- 172.21.15.143:0/4112005601 shutdown_connections 2024-01-16T15:02:17.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:17.551+0000 7f268f200640 1 -- 172.21.15.143:0/4112005601 wait complete. 2024-01-16T15:02:17.580 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-16T15:02:18.581 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-16T15:02:18.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.809+0000 7f36f9a00640 1 Processor -- start 2024-01-16T15:02:18.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.810+0000 7f36f9a00640 1 -- start start 2024-01-16T15:02:18.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.811+0000 7f36f9a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36f40c49f0 0x7f36f415e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:18.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.811+0000 7f36f9a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36f415ec90 con 0x7f36f4072160 2024-01-16T15:02:18.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.811+0000 7f36f9a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36f40cd530 con 0x7f36f406de10 2024-01-16T15:02:18.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.811+0000 7f36f9a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36f40c4ed0 con 0x7f36f40c49f0 2024-01-16T15:02:18.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.811+0000 7f36f2a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f36f4072160 0x7f36f406d700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49018/0 (socket says 172.21.15.143:49018) 2024-01-16T15:02:18.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.811+0000 7f36f2a00640 1 -- 172.21.15.143:0/2719087864 learned_addr learned my addr 172.21.15.143:0/2719087864 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:02:18.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.812+0000 7f36f3400640 1 --2- 172.21.15.143:0/2719087864 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36f40c49f0 0x7f36f415e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:18.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.812+0000 7f36f1600640 1 -- 172.21.15.143:0/2719087864 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2677262669 0 0) 0x7f36e4001110 con 0x7f36f4072160 2024-01-16T15:02:18.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.812+0000 7f36f1600640 1 -- 172.21.15.143:0/2719087864 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36d4005730 con 0x7f36f4072160 2024-01-16T15:02:18.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.812+0000 7f36f1600640 1 -- 172.21.15.143:0/2719087864 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1486128856 0 0) 0x7f36d4005730 con 0x7f36f4072160 2024-01-16T15:02:18.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.812+0000 7f36f1600640 1 -- 172.21.15.143:0/2719087864 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36f40c49f0 msgr2=0x7f36f415e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:18.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.812+0000 7f36f1600640 1 --2- 172.21.15.143:0/2719087864 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36f40c49f0 0x7f36f415e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:18.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.812+0000 7f36f1600640 1 -- 172.21.15.143:0/2719087864 >> v1:172.21.15.204:6789/0 conn(0x7f36f406de10 legacy=0x7f36f40705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:18.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.812+0000 7f36f1600640 1 -- 172.21.15.143:0/2719087864 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36f40c51b0 con 0x7f36f4072160 2024-01-16T15:02:18.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.812+0000 7f36f1600640 1 -- 172.21.15.143:0/2719087864 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7f36e4001800 con 0x7f36f4072160 2024-01-16T15:02:18.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.812+0000 7f36f1600640 1 -- 172.21.15.143:0/2719087864 >> v1:172.21.15.143:6789/0 conn(0x7f36f4072160 legacy=0x7f36f406d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:18.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.812+0000 7f36f1600640 1 --2- 172.21.15.143:0/2719087864 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36f40c49f0 0x7f36d4005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:18.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.812+0000 7f36f1600640 1 --2- 172.21.15.143:0/2719087864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36d4006130 0x7f36d40069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:18.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.812+0000 7f36f1600640 1 --2- 172.21.15.143:0/2719087864 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36d4006f10 0x7f36d4007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:18.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.812+0000 7f36f1600640 1 -- 172.21.15.143:0/2719087864 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f40c4ed0 con 0x7f36d4006f10 2024-01-16T15:02:18.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.812+0000 7f36f1600640 1 -- 172.21.15.143:0/2719087864 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d400dc70 con 0x7f36d4006130 2024-01-16T15:02:18.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.812+0000 7f36f1600640 1 -- 172.21.15.143:0/2719087864 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36d400de30 con 0x7f36f40c49f0 2024-01-16T15:02:18.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.812+0000 7f36f3400640 1 --2- 172.21.15.143:0/2719087864 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36d4006f10 0x7f36d4007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:18.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.813+0000 7f36f2a00640 1 --2- 172.21.15.143:0/2719087864 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36f40c49f0 0x7f36d4005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:18.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.813+0000 7f36f3400640 1 -- 172.21.15.143:0/2719087864 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36f40c49f0 msgr2=0x7f36d4005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:18.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.813+0000 7f36f3400640 1 --2- 172.21.15.143:0/2719087864 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36f40c49f0 0x7f36d4005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:18.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.813+0000 7f36f2000640 1 --2- 172.21.15.143:0/2719087864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36d4006130 0x7f36d40069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:18.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.813+0000 7f36f3400640 1 -- 172.21.15.143:0/2719087864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36d4006130 msgr2=0x7f36d40069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:18.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.813+0000 7f36f3400640 1 --2- 172.21.15.143:0/2719087864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36d4006130 0x7f36d40069d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:18.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.813+0000 7f36f3400640 1 -- 172.21.15.143:0/2719087864 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d4005730 con 0x7f36d4006f10 2024-01-16T15:02:18.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.814+0000 7f36f3400640 1 --2- 172.21.15.143:0/2719087864 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36d4006f10 0x7f36d4007390 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f36f406ead0 tx=0x7f36d4007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=746c27abff923b62 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:18.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.814+0000 7f36f1600640 1 -- 172.21.15.143:0/2719087864 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ec02cb90 con 0x7f36d4006f10 2024-01-16T15:02:18.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.814+0000 7f36f1600640 1 -- 172.21.15.143:0/2719087864 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36ec00f040 con 0x7f36d4006f10 2024-01-16T15:02:18.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.814+0000 7f36f1600640 1 -- 172.21.15.143:0/2719087864 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ec034440 con 0x7f36d4006f10 2024-01-16T15:02:18.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.815+0000 7f36f9a00640 1 -- 172.21.15.143:0/2719087864 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36d4006f10 msgr2=0x7f36d4007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:18.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.815+0000 7f36f9a00640 1 --2- 172.21.15.143:0/2719087864 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36d4006f10 0x7f36d4007390 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f36f406ead0 tx=0x7f36d4007a00 comp rx=0 tx=0).stop 2024-01-16T15:02:18.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.815+0000 7f36f3400640 1 -- 172.21.15.143:0/2719087864 reap_dead start 2024-01-16T15:02:18.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.815+0000 7f36f9a00640 1 -- 172.21.15.143:0/2719087864 shutdown_connections 2024-01-16T15:02:18.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.815+0000 7f36f9a00640 1 -- 172.21.15.143:0/2719087864 >> 172.21.15.143:0/2719087864 conn(0x7f36f40b0c50 msgr2=0x7f36f40b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:18.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.815+0000 7f36f9a00640 1 -- 172.21.15.143:0/2719087864 shutdown_connections 2024-01-16T15:02:18.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.815+0000 7f36f9a00640 1 -- 172.21.15.143:0/2719087864 wait complete. 2024-01-16T15:02:18.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.817+0000 7f36f9a00640 1 Processor -- start 2024-01-16T15:02:18.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.817+0000 7f36f9a00640 1 -- start start 2024-01-16T15:02:18.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.817+0000 7f36f9a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36d4006f10 0x7f36f41567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:18.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.818+0000 7f36f9a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36f41570a0 0x7f36f40c60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:18.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.818+0000 7f36f9a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36f40c7ae0 0x7f36f40c8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:18.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.818+0000 7f36f9a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f40c6a40 con 0x7f36f40c7ae0 2024-01-16T15:02:18.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.818+0000 7f36f9a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f40c6d20 con 0x7f36f41570a0 2024-01-16T15:02:18.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.818+0000 7f36f9a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36f40c7000 con 0x7f36d4006f10 2024-01-16T15:02:18.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.818+0000 7f36f3400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36f40c7ae0 0x7f36f40c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:18.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.818+0000 7f36f3400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36f40c7ae0 0x7f36f40c8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34328/0 (socket says 172.21.15.143:34328) 2024-01-16T15:02:18.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.818+0000 7f36f3400640 1 -- 172.21.15.143:0/569270943 learned_addr learned my addr 172.21.15.143:0/569270943 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:02:18.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.818+0000 7f36f2a00640 1 --2- 172.21.15.143:0/569270943 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36d4006f10 0x7f36f41567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:18.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.818+0000 7f36f3400640 1 -- 172.21.15.143:0/569270943 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36d4006f10 msgr2=0x7f36f41567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:18.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.818+0000 7f36f3400640 1 --2- 172.21.15.143:0/569270943 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36d4006f10 0x7f36f41567c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:18.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.818+0000 7f36f3400640 1 -- 172.21.15.143:0/569270943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36f41570a0 msgr2=0x7f36f40c60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:18.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.818+0000 7f36f3400640 1 --2- 172.21.15.143:0/569270943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36f41570a0 0x7f36f40c60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:18.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.818+0000 7f36f3400640 1 -- 172.21.15.143:0/569270943 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36f40c72e0 con 0x7f36f40c7ae0 2024-01-16T15:02:18.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.818+0000 7f36f3400640 1 --2- 172.21.15.143:0/569270943 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36f40c7ae0 0x7f36f40c8710 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f36ec037040 tx=0x7f36ec009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:18.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.819+0000 7f36eaa00640 1 -- 172.21.15.143:0/569270943 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ec034cd0 con 0x7f36f40c7ae0 2024-01-16T15:02:18.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.819+0000 7f36eaa00640 1 -- 172.21.15.143:0/569270943 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36ec034e30 con 0x7f36f40c7ae0 2024-01-16T15:02:18.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.819+0000 7f36f9a00640 1 -- 172.21.15.143:0/569270943 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36f40c75c0 con 0x7f36f40c7ae0 2024-01-16T15:02:18.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.819+0000 7f36eaa00640 1 -- 172.21.15.143:0/569270943 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ec03e660 con 0x7f36f40c7ae0 2024-01-16T15:02:18.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.819+0000 7f36f9a00640 1 -- 172.21.15.143:0/569270943 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36f40c6610 con 0x7f36f40c7ae0 2024-01-16T15:02:18.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.820+0000 7f36f9a00640 1 -- 172.21.15.143:0/569270943 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36f40c6800 con 0x7f36f40c7ae0 2024-01-16T15:02:18.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.821+0000 7f36eaa00640 1 -- 172.21.15.143:0/569270943 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7f36ec03e7c0 con 0x7f36f40c7ae0 2024-01-16T15:02:18.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.822+0000 7f36eaa00640 1 --2- 172.21.15.143:0/569270943 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f36cc06b470 0x7f36cc06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:18.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.822+0000 7f36eaa00640 1 -- 172.21.15.143:0/569270943 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f36ec0abe50 con 0x7f36f40c7ae0 2024-01-16T15:02:18.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.822+0000 7f36f2a00640 1 --2- 172.21.15.143:0/569270943 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f36cc06b470 0x7f36cc06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:18.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.822+0000 7f36f2a00640 1 --2- 172.21.15.143:0/569270943 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f36cc06b470 0x7f36cc06d930 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f36e4001e10 tx=0x7f36e4001110 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:18.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:18.826+0000 7f36eaa00640 1 -- 172.21.15.143:0/569270943 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f36f40c6800 con 0x7f36f40c7ae0 2024-01-16T15:02:19.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.036+0000 7f36f9a00640 1 -- 172.21.15.143:0/569270943 --> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f36f40c6e90 con 0x7f36cc06b470 2024-01-16T15:02:19.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.496+0000 7f36eaa00640 1 -- 172.21.15.143:0/569270943 <== mgr.4113 v2:172.21.15.143:6832/3153492115 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7f36f40c6e90 con 0x7f36cc06b470 2024-01-16T15:02:19.477 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-16T15:02:19.477 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-16T15:02:19.477 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "complete" 2024-01-16T15:02:19.477 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:02:19.477 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-16T15:02:19.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.500+0000 7f36f9a00640 1 -- 172.21.15.143:0/569270943 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f36cc06b470 msgr2=0x7f36cc06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:19.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.500+0000 7f36f9a00640 1 --2- 172.21.15.143:0/569270943 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f36cc06b470 0x7f36cc06d930 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f36e4001e10 tx=0x7f36e4001110 comp rx=0 tx=0).stop 2024-01-16T15:02:19.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.500+0000 7f36f9a00640 1 -- 172.21.15.143:0/569270943 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36f40c7ae0 msgr2=0x7f36f40c8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:19.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.500+0000 7f36f9a00640 1 --2- 172.21.15.143:0/569270943 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36f40c7ae0 0x7f36f40c8710 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f36ec037040 tx=0x7f36ec009750 comp rx=0 tx=0).stop 2024-01-16T15:02:19.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.500+0000 7f36f9a00640 1 -- 172.21.15.143:0/569270943 shutdown_connections 2024-01-16T15:02:19.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.500+0000 7f36f9a00640 1 --2- 172.21.15.143:0/569270943 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f36cc06b470 0x7f36cc06d930 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:19.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.500+0000 7f36f9a00640 1 --2- 172.21.15.143:0/569270943 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36f40c7ae0 0x7f36f40c8710 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:19.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.500+0000 7f36f9a00640 1 --2- 172.21.15.143:0/569270943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36f41570a0 0x7f36f40c60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:19.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.500+0000 7f36f9a00640 1 --2- 172.21.15.143:0/569270943 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36d4006f10 0x7f36f41567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:19.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.500+0000 7f36f9a00640 1 -- 172.21.15.143:0/569270943 >> 172.21.15.143:0/569270943 conn(0x7f36f40b0c50 msgr2=0x7f36f40acca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:19.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.500+0000 7f36f9a00640 1 -- 172.21.15.143:0/569270943 shutdown_connections 2024-01-16T15:02:19.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.500+0000 7f36f9a00640 1 -- 172.21.15.143:0/569270943 wait complete. 2024-01-16T15:02:19.501 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-16T15:02:19.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.731+0000 7fbf40e00640 1 Processor -- start 2024-01-16T15:02:19.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.732+0000 7fbf40e00640 1 -- start start 2024-01-16T15:02:19.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.733+0000 7fbf40e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbf3c15b440 0x7fbf3c0cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:19.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.733+0000 7fbf40e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf3c0c5dc0 con 0x7fbf3c0cbf20 2024-01-16T15:02:19.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.733+0000 7fbf40e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf3c15eea0 con 0x7fbf3c0c59b0 2024-01-16T15:02:19.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.733+0000 7fbf40e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf3c0cc3d0 con 0x7fbf3c15b440 2024-01-16T15:02:19.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.733+0000 7fbf39e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbf3c15b440 0x7fbf3c0cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:19.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.734+0000 7fbf39e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbf3c15b440 0x7fbf3c0cb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34342/0 (socket says 172.21.15.143:34342) 2024-01-16T15:02:19.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.734+0000 7fbf39e00640 1 -- 172.21.15.143:0/824105917 learned_addr learned my addr 172.21.15.143:0/824105917 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:02:19.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.734+0000 7fbf39e00640 1 -- 172.21.15.143:0/824105917 >> v1:172.21.15.204:6789/0 conn(0x7fbf3c0c59b0 legacy=0x7fbf3c06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:19.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.734+0000 7fbf39e00640 1 -- 172.21.15.143:0/824105917 >> v1:172.21.15.143:6789/0 conn(0x7fbf3c0cbf20 legacy=0x7fbf3c0c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:19.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.734+0000 7fbf39e00640 1 -- 172.21.15.143:0/824105917 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf3c0abf30 con 0x7fbf3c15b440 2024-01-16T15:02:19.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.734+0000 7fbf39400640 1 -- 172.21.15.143:0/824105917 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3423268019 0 0) 0x7fbf3c0c5dc0 con 0x7fbf3c0cbf20 2024-01-16T15:02:19.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.736+0000 7fbf39e00640 1 --2- 172.21.15.143:0/824105917 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf3c15b440 0x7fbf3c0cb9e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c009870 tx=0x7fbf2c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ed17c757b76d60 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:19.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.736+0000 7fbf39400640 1 -- 172.21.15.143:0/824105917 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c030860 con 0x7fbf3c15b440 2024-01-16T15:02:19.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.736+0000 7fbf39400640 1 -- 172.21.15.143:0/824105917 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf2c0309c0 con 0x7fbf3c15b440 2024-01-16T15:02:19.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.736+0000 7fbf39400640 1 -- 172.21.15.143:0/824105917 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c030d50 con 0x7fbf3c15b440 2024-01-16T15:02:19.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.737+0000 7fbf40e00640 1 -- 172.21.15.143:0/824105917 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf3c15b440 msgr2=0x7fbf3c0cb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:19.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.737+0000 7fbf40e00640 1 --2- 172.21.15.143:0/824105917 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf3c15b440 0x7fbf3c0cb9e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c009870 tx=0x7fbf2c02b9a0 comp rx=0 tx=0).stop 2024-01-16T15:02:19.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.737+0000 7fbf40e00640 1 -- 172.21.15.143:0/824105917 shutdown_connections 2024-01-16T15:02:19.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.737+0000 7fbf40e00640 1 --2- 172.21.15.143:0/824105917 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf3c15b440 0x7fbf3c0cb9e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:19.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.737+0000 7fbf40e00640 1 -- 172.21.15.143:0/824105917 >> 172.21.15.143:0/824105917 conn(0x7fbf3c0b0c50 msgr2=0x7fbf3c0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:19.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.737+0000 7fbf40e00640 1 -- 172.21.15.143:0/824105917 shutdown_connections 2024-01-16T15:02:19.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.737+0000 7fbf40e00640 1 -- 172.21.15.143:0/824105917 wait complete. 2024-01-16T15:02:19.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.739+0000 7fbf40e00640 1 Processor -- start 2024-01-16T15:02:19.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.739+0000 7fbf40e00640 1 -- start start 2024-01-16T15:02:19.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.740+0000 7fbf40e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf3c0c59b0 0x7fbf3c15c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:19.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.740+0000 7fbf40e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf3c0cbf20 0x7fbf3c0b4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:19.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.740+0000 7fbf40e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf3c15b440 0x7fbf3c15d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:19.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.740+0000 7fbf40e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c15b9a0 con 0x7fbf3c15b440 2024-01-16T15:02:19.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.740+0000 7fbf40e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c15db20 con 0x7fbf3c0cbf20 2024-01-16T15:02:19.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.740+0000 7fbf40e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c15de00 con 0x7fbf3c0c59b0 2024-01-16T15:02:19.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.740+0000 7fbf3a800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf3c15b440 0x7fbf3c15d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:19.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.740+0000 7fbf3a800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf3c15b440 0x7fbf3c15d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34348/0 (socket says 172.21.15.143:34348) 2024-01-16T15:02:19.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.740+0000 7fbf3a800640 1 -- 172.21.15.143:0/2918739098 learned_addr learned my addr 172.21.15.143:0/2918739098 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:02:19.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.740+0000 7fbf31200640 1 --2- 172.21.15.143:0/2918739098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf3c0cbf20 0x7fbf3c0b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:19.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.740+0000 7fbf3a800640 1 -- 172.21.15.143:0/2918739098 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf3c0c59b0 msgr2=0x7fbf3c15c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:19.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.740+0000 7fbf39e00640 1 --2- 172.21.15.143:0/2918739098 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf3c0c59b0 0x7fbf3c15c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:19.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.740+0000 7fbf3a800640 1 --2- 172.21.15.143:0/2918739098 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf3c0c59b0 0x7fbf3c15c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:19.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.741+0000 7fbf3a800640 1 -- 172.21.15.143:0/2918739098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf3c0cbf20 msgr2=0x7fbf3c0b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:19.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.741+0000 7fbf3a800640 1 --2- 172.21.15.143:0/2918739098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf3c0cbf20 0x7fbf3c0b4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:19.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.741+0000 7fbf3a800640 1 -- 172.21.15.143:0/2918739098 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf2c0094d0 con 0x7fbf3c15b440 2024-01-16T15:02:19.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.741+0000 7fbf3a800640 1 --2- 172.21.15.143:0/2918739098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf3c15b440 0x7fbf3c15d060 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbf3400c7a0 tx=0x7fbf3400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:19.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.741+0000 7fbf32a00640 1 -- 172.21.15.143:0/2918739098 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf34003b70 con 0x7fbf3c15b440 2024-01-16T15:02:19.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.741+0000 7fbf32a00640 1 -- 172.21.15.143:0/2918739098 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf34003cf0 con 0x7fbf3c15b440 2024-01-16T15:02:19.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.741+0000 7fbf32a00640 1 -- 172.21.15.143:0/2918739098 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf34003ea0 con 0x7fbf3c15b440 2024-01-16T15:02:19.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.741+0000 7fbf40e00640 1 -- 172.21.15.143:0/2918739098 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf3c15bbf0 con 0x7fbf3c15b440 2024-01-16T15:02:19.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.741+0000 7fbf40e00640 1 -- 172.21.15.143:0/2918739098 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf3c156e30 con 0x7fbf3c15b440 2024-01-16T15:02:19.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.742+0000 7fbf40e00640 1 -- 172.21.15.143:0/2918739098 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf3c15e020 con 0x7fbf3c15b440 2024-01-16T15:02:19.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.743+0000 7fbf32a00640 1 -- 172.21.15.143:0/2918739098 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7fbf3400aa90 con 0x7fbf3c15b440 2024-01-16T15:02:19.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.744+0000 7fbf32a00640 1 --2- 172.21.15.143:0/2918739098 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbf1406b470 0x7fbf1406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:19.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.744+0000 7fbf32a00640 1 -- 172.21.15.143:0/2918739098 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbf34089de0 con 0x7fbf3c15b440 2024-01-16T15:02:19.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.744+0000 7fbf39e00640 1 --2- 172.21.15.143:0/2918739098 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbf1406b470 0x7fbf1406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:19.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.745+0000 7fbf39e00640 1 --2- 172.21.15.143:0/2918739098 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbf1406b470 0x7fbf1406d930 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c036000 tx=0x7fbf2c03b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:19.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.748+0000 7fbf32a00640 1 -- 172.21.15.143:0/2918739098 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbf34054930 con 0x7fbf3c15b440 2024-01-16T15:02:19.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.964+0000 7fbf40e00640 1 -- 172.21.15.143:0/2918739098 --> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf3c15f3d0 con 0x7fbf1406b470 2024-01-16T15:02:19.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.967+0000 7fbf32a00640 1 -- 172.21.15.143:0/2918739098 <== mgr.4113 v2:172.21.15.143:6832/3153492115 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fbf2c030bb0 con 0x7fbf1406b470 2024-01-16T15:02:19.948 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-16T15:02:19.948 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-16T15:02:19.948 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cephfs" 2024-01-16T15:02:19.948 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:02:19.948 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-16T15:02:19.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.970+0000 7fbf40e00640 1 -- 172.21.15.143:0/2918739098 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbf1406b470 msgr2=0x7fbf1406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:19.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.970+0000 7fbf40e00640 1 --2- 172.21.15.143:0/2918739098 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbf1406b470 0x7fbf1406d930 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c036000 tx=0x7fbf2c03b040 comp rx=0 tx=0).stop 2024-01-16T15:02:19.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.970+0000 7fbf40e00640 1 -- 172.21.15.143:0/2918739098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf3c15b440 msgr2=0x7fbf3c15d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:19.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.970+0000 7fbf40e00640 1 --2- 172.21.15.143:0/2918739098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf3c15b440 0x7fbf3c15d060 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbf3400c7a0 tx=0x7fbf3400cc60 comp rx=0 tx=0).stop 2024-01-16T15:02:19.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.970+0000 7fbf40e00640 1 -- 172.21.15.143:0/2918739098 shutdown_connections 2024-01-16T15:02:19.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.970+0000 7fbf40e00640 1 --2- 172.21.15.143:0/2918739098 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fbf1406b470 0x7fbf1406d930 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:19.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.970+0000 7fbf40e00640 1 --2- 172.21.15.143:0/2918739098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf3c15b440 0x7fbf3c15d060 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:19.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.970+0000 7fbf40e00640 1 --2- 172.21.15.143:0/2918739098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf3c0cbf20 0x7fbf3c0b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:19.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.970+0000 7fbf40e00640 1 --2- 172.21.15.143:0/2918739098 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf3c0c59b0 0x7fbf3c15c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:19.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.970+0000 7fbf40e00640 1 -- 172.21.15.143:0/2918739098 >> 172.21.15.143:0/2918739098 conn(0x7fbf3c0b0c50 msgr2=0x7fbf3c071e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:19.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.970+0000 7fbf40e00640 1 -- 172.21.15.143:0/2918739098 shutdown_connections 2024-01-16T15:02:19.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:19.970+0000 7fbf40e00640 1 -- 172.21.15.143:0/2918739098 wait complete. 2024-01-16T15:02:19.969 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-01-16T15:02:20.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.199+0000 7f7f37e00640 1 Processor -- start 2024-01-16T15:02:20.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.199+0000 7f7f37e00640 1 -- start start 2024-01-16T15:02:20.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.200+0000 7f7f37e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7f381609f0 0x7f7f380d12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:20.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.200+0000 7f7f37e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7f38160dd0 0x7f7f380d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:20.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.200+0000 7f7f37e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f380b41b0 con 0x7f7f380b2050 2024-01-16T15:02:20.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.200+0000 7f7f37e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f380b4490 con 0x7f7f38160dd0 2024-01-16T15:02:20.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.200+0000 7f7f37e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f380b0090 con 0x7f7f381609f0 2024-01-16T15:02:20.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.201+0000 7f7f2fe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7f38160dd0 0x7f7f380d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:20.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.201+0000 7f7f2fe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7f38160dd0 0x7f7f380d1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34354/0 (socket says 172.21.15.143:34354) 2024-01-16T15:02:20.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.201+0000 7f7f2fe00640 1 -- 172.21.15.143:0/1352749987 learned_addr learned my addr 172.21.15.143:0/1352749987 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:02:20.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.201+0000 7f7f36a00640 1 --2- 172.21.15.143:0/1352749987 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7f381609f0 0x7f7f380d12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:20.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.201+0000 7f7f2fe00640 1 -- 172.21.15.143:0/1352749987 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7f381609f0 msgr2=0x7f7f380d12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:20.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.201+0000 7f7f2fe00640 1 --2- 172.21.15.143:0/1352749987 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7f381609f0 0x7f7f380d12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:20.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.201+0000 7f7f2fe00640 1 -- 172.21.15.143:0/1352749987 >> v1:172.21.15.204:6789/0 conn(0x7f7f380b2050 legacy=0x7f7f380b0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:20.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.201+0000 7f7f2fe00640 1 -- 172.21.15.143:0/1352749987 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f380b0370 con 0x7f7f38160dd0 2024-01-16T15:02:20.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.201+0000 7f7f36a00640 1 --2- 172.21.15.143:0/1352749987 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7f381609f0 0x7f7f380d12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:02:20.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.202+0000 7f7f2fe00640 1 --2- 172.21.15.143:0/1352749987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f38160dd0 0x7f7f380d1830 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7f20009870 tx=0x7f7f2002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94677cba5abfe12e server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:20.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.203+0000 7f7f36000640 1 -- 172.21.15.143:0/1352749987 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f20031640 con 0x7f7f38160dd0 2024-01-16T15:02:20.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.203+0000 7f7f36000640 1 -- 172.21.15.143:0/1352749987 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f200317a0 con 0x7f7f38160dd0 2024-01-16T15:02:20.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.203+0000 7f7f36000640 1 -- 172.21.15.143:0/1352749987 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f20031b30 con 0x7f7f38160dd0 2024-01-16T15:02:20.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.203+0000 7f7f37e00640 1 -- 172.21.15.143:0/1352749987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f38160dd0 msgr2=0x7f7f380d1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:20.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.203+0000 7f7f37e00640 1 --2- 172.21.15.143:0/1352749987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f38160dd0 0x7f7f380d1830 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7f20009870 tx=0x7f7f2002b8f0 comp rx=0 tx=0).stop 2024-01-16T15:02:20.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.204+0000 7f7f37e00640 1 -- 172.21.15.143:0/1352749987 shutdown_connections 2024-01-16T15:02:20.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.204+0000 7f7f37e00640 1 --2- 172.21.15.143:0/1352749987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f38160dd0 0x7f7f380d1830 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:20.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.204+0000 7f7f37e00640 1 --2- 172.21.15.143:0/1352749987 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7f381609f0 0x7f7f380d12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:20.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.204+0000 7f7f37e00640 1 -- 172.21.15.143:0/1352749987 >> 172.21.15.143:0/1352749987 conn(0x7f7f3809c5b0 msgr2=0x7f7f380d3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:20.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.204+0000 7f7f37e00640 1 -- 172.21.15.143:0/1352749987 shutdown_connections 2024-01-16T15:02:20.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.204+0000 7f7f37e00640 1 -- 172.21.15.143:0/1352749987 wait complete. 2024-01-16T15:02:20.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.205+0000 7f7f37e00640 1 Processor -- start 2024-01-16T15:02:20.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.206+0000 7f7f37e00640 1 -- start start 2024-01-16T15:02:20.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.207+0000 7f7f37e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f380b2050 0x7f7f381f3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:20.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.207+0000 7f7f37e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7f381609f0 0x7f7f380ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:20.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.207+0000 7f7f37e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f38160dd0 0x7f7f38156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:20.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.207+0000 7f7f37e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f381f41a0 con 0x7f7f38160dd0 2024-01-16T15:02:20.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.207+0000 7f7f37e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f380cbc70 con 0x7f7f380b2050 2024-01-16T15:02:20.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.207+0000 7f7f37e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f380cbf50 con 0x7f7f381609f0 2024-01-16T15:02:20.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.207+0000 7f7f37400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f38160dd0 0x7f7f38156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:20.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.207+0000 7f7f37400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f38160dd0 0x7f7f38156e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34364/0 (socket says 172.21.15.143:34364) 2024-01-16T15:02:20.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.207+0000 7f7f37400640 1 -- 172.21.15.143:0/998157844 learned_addr learned my addr 172.21.15.143:0/998157844 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:02:20.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.207+0000 7f7f36a00640 1 --2- 172.21.15.143:0/998157844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f380b2050 0x7f7f381f3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:20.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.207+0000 7f7f37400640 1 -- 172.21.15.143:0/998157844 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7f381609f0 msgr2=0x7f7f380ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:20.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.207+0000 7f7f37400640 1 --2- 172.21.15.143:0/998157844 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7f381609f0 0x7f7f380ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:20.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.207+0000 7f7f37400640 1 -- 172.21.15.143:0/998157844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f380b2050 msgr2=0x7f7f381f3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:20.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.207+0000 7f7f37400640 1 --2- 172.21.15.143:0/998157844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f380b2050 0x7f7f381f3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:20.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.207+0000 7f7f37400640 1 -- 172.21.15.143:0/998157844 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f200094d0 con 0x7f7f38160dd0 2024-01-16T15:02:20.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.207+0000 7f7f36a00640 1 --2- 172.21.15.143:0/998157844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f380b2050 0x7f7f381f3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:02:20.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.208+0000 7f7f37400640 1 --2- 172.21.15.143:0/998157844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f38160dd0 0x7f7f38156e50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000c700 tx=0x7f7f3000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:20.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.208+0000 7f7f2ea00640 1 -- 172.21.15.143:0/998157844 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3000aa50 con 0x7f7f38160dd0 2024-01-16T15:02:20.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.208+0000 7f7f37e00640 1 -- 172.21.15.143:0/998157844 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f380cc230 con 0x7f7f38160dd0 2024-01-16T15:02:20.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.208+0000 7f7f2ea00640 1 -- 172.21.15.143:0/998157844 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f3000abb0 con 0x7f7f38160dd0 2024-01-16T15:02:20.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.208+0000 7f7f2ea00640 1 -- 172.21.15.143:0/998157844 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3000ce10 con 0x7f7f38160dd0 2024-01-16T15:02:20.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.208+0000 7f7f37e00640 1 -- 172.21.15.143:0/998157844 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f380cc7f0 con 0x7f7f38160dd0 2024-01-16T15:02:20.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.209+0000 7f7f37e00640 1 -- 172.21.15.143:0/998157844 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f380cd310 con 0x7f7f38160dd0 2024-01-16T15:02:20.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.211+0000 7f7f2ea00640 1 -- 172.21.15.143:0/998157844 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7f7f30012420 con 0x7f7f38160dd0 2024-01-16T15:02:20.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.212+0000 7f7f2ea00640 1 --2- 172.21.15.143:0/998157844 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f7f1006b470 0x7f7f1006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:20.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.212+0000 7f7f2ea00640 1 -- 172.21.15.143:0/998157844 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f3008a160 con 0x7f7f38160dd0 2024-01-16T15:02:20.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.212+0000 7f7f36a00640 1 --2- 172.21.15.143:0/998157844 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f7f1006b470 0x7f7f1006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:20.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.213+0000 7f7f36a00640 1 --2- 172.21.15.143:0/998157844 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f7f1006b470 0x7f7f1006d930 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7f28006fd0 tx=0x7f7f28008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:20.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.216+0000 7f7f2ea00640 1 -- 172.21.15.143:0/998157844 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7f30054cc0 con 0x7f7f38160dd0 2024-01-16T15:02:20.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.434+0000 7f7f37e00640 1 -- 172.21.15.143:0/998157844 --> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7f380d29a0 con 0x7f7f1006b470 2024-01-16T15:02:20.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.440+0000 7f7f2ea00640 1 -- 172.21.15.143:0/998157844 <== mgr.4113 v2:172.21.15.143:6832/3153492115 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+79 (secure 0 0 0) 0x7f7f380d29a0 con 0x7f7f1006b470 2024-01-16T15:02:20.421 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:02:20.421 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-16T15:02:20.421 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-16T15:02:20.421 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "sv1" 2024-01-16T15:02:20.421 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-16T15:02:20.421 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-16T15:02:20.421 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ss1clone1" 2024-01-16T15:02:20.421 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:02:20.421 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-16T15:02:20.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.442+0000 7f7f37e00640 1 -- 172.21.15.143:0/998157844 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f7f1006b470 msgr2=0x7f7f1006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:20.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.443+0000 7f7f37e00640 1 --2- 172.21.15.143:0/998157844 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f7f1006b470 0x7f7f1006d930 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7f28006fd0 tx=0x7f7f28008040 comp rx=0 tx=0).stop 2024-01-16T15:02:20.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.443+0000 7f7f37e00640 1 -- 172.21.15.143:0/998157844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f38160dd0 msgr2=0x7f7f38156e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:20.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.443+0000 7f7f37e00640 1 --2- 172.21.15.143:0/998157844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f38160dd0 0x7f7f38156e50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000c700 tx=0x7f7f3000cbc0 comp rx=0 tx=0).stop 2024-01-16T15:02:20.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.443+0000 7f7f37e00640 1 -- 172.21.15.143:0/998157844 shutdown_connections 2024-01-16T15:02:20.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.443+0000 7f7f37e00640 1 --2- 172.21.15.143:0/998157844 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f7f1006b470 0x7f7f1006d930 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:20.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.443+0000 7f7f37e00640 1 --2- 172.21.15.143:0/998157844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f38160dd0 0x7f7f38156e50 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:20.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.443+0000 7f7f37e00640 1 --2- 172.21.15.143:0/998157844 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7f381609f0 0x7f7f380ccd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:20.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.443+0000 7f7f37e00640 1 --2- 172.21.15.143:0/998157844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f380b2050 0x7f7f381f3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:20.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.443+0000 7f7f37e00640 1 -- 172.21.15.143:0/998157844 >> 172.21.15.143:0/998157844 conn(0x7f7f3809c5b0 msgr2=0x7f7f3806b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:20.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.443+0000 7f7f37e00640 1 -- 172.21.15.143:0/998157844 shutdown_connections 2024-01-16T15:02:20.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.443+0000 7f7f37e00640 1 -- 172.21.15.143:0/998157844 wait complete. 2024-01-16T15:02:20.441 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-01-16T15:02:20.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.667+0000 7f5795000640 1 Processor -- start 2024-01-16T15:02:20.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.668+0000 7f5795000640 1 -- start start 2024-01-16T15:02:20.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.669+0000 7f5795000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f57900af440 0x7f57900b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:20.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.669+0000 7f5795000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57900cf500 0x7f57900cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:20.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.669+0000 7f5795000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57900afb60 con 0x7f57900b1790 2024-01-16T15:02:20.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.669+0000 7f5795000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57900b1b70 con 0x7f57900af440 2024-01-16T15:02:20.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.669+0000 7f5795000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57900b0120 con 0x7f57900cf500 2024-01-16T15:02:20.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.669+0000 7f578d400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f57900af440 0x7f57900b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:20.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.669+0000 7f578d400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f57900af440 0x7f57900b05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34378/0 (socket says 172.21.15.143:34378) 2024-01-16T15:02:20.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.669+0000 7f578d400640 1 -- 172.21.15.143:0/3414291316 learned_addr learned my addr 172.21.15.143:0/3414291316 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:02:20.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.669+0000 7f578e800640 1 --2- 172.21.15.143:0/3414291316 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57900cf500 0x7f57900cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:20.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.670+0000 7f578d400640 1 -- 172.21.15.143:0/3414291316 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57900cf500 msgr2=0x7f57900cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:20.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.670+0000 7f578d400640 1 --2- 172.21.15.143:0/3414291316 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57900cf500 0x7f57900cf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:20.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.670+0000 7f578d400640 1 -- 172.21.15.143:0/3414291316 >> v1:172.21.15.204:6789/0 conn(0x7f57900b1790 legacy=0x7f579006a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:20.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.670+0000 7f578d400640 1 -- 172.21.15.143:0/3414291316 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57900b0af0 con 0x7f57900af440 2024-01-16T15:02:20.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.671+0000 7f578d400640 1 --2- 172.21.15.143:0/3414291316 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57900af440 0x7f57900b05b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5778009e50 tx=0x7f577802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81809231b9fbe730 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:20.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.671+0000 7f578ca00640 1 -- 172.21.15.143:0/3414291316 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577802c9e0 con 0x7f57900af440 2024-01-16T15:02:20.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.671+0000 7f578ca00640 1 -- 172.21.15.143:0/3414291316 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f577802cb40 con 0x7f57900af440 2024-01-16T15:02:20.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.671+0000 7f578ca00640 1 -- 172.21.15.143:0/3414291316 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57780343f0 con 0x7f57900af440 2024-01-16T15:02:20.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.671+0000 7f5795000640 1 -- 172.21.15.143:0/3414291316 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57900af440 msgr2=0x7f57900b05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:20.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.671+0000 7f5795000640 1 --2- 172.21.15.143:0/3414291316 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57900af440 0x7f57900b05b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5778009e50 tx=0x7f577802be40 comp rx=0 tx=0).stop 2024-01-16T15:02:20.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.672+0000 7f5795000640 1 -- 172.21.15.143:0/3414291316 shutdown_connections 2024-01-16T15:02:20.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.672+0000 7f5795000640 1 --2- 172.21.15.143:0/3414291316 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57900cf500 0x7f57900cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:20.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.672+0000 7f5795000640 1 --2- 172.21.15.143:0/3414291316 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57900af440 0x7f57900b05b0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:20.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.672+0000 7f5795000640 1 -- 172.21.15.143:0/3414291316 >> 172.21.15.143:0/3414291316 conn(0x7f57900b1cf0 msgr2=0x7f57900b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:20.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.672+0000 7f5795000640 1 -- 172.21.15.143:0/3414291316 shutdown_connections 2024-01-16T15:02:20.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.672+0000 7f5795000640 1 -- 172.21.15.143:0/3414291316 wait complete. 2024-01-16T15:02:20.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.673+0000 7f5795000640 1 Processor -- start 2024-01-16T15:02:20.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.673+0000 7f5795000640 1 -- start start 2024-01-16T15:02:20.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.674+0000 7f5795000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57900af440 0x7f579009db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:20.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.674+0000 7f5795000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57900b1790 0x7f57900b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:20.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.674+0000 7f5795000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57900cf500 0x7f579015f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:20.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.674+0000 7f5795000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57901e89c0 con 0x7f57900b1790 2024-01-16T15:02:20.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.674+0000 7f5795000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57901e7d40 con 0x7f57900af440 2024-01-16T15:02:20.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.674+0000 7f5795000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57901e8020 con 0x7f57900cf500 2024-01-16T15:02:20.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.675+0000 7f578d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57900b1790 0x7f57900b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:20.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.675+0000 7f578d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57900b1790 0x7f57900b4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34392/0 (socket says 172.21.15.143:34392) 2024-01-16T15:02:20.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.675+0000 7f578d400640 1 -- 172.21.15.143:0/189369405 learned_addr learned my addr 172.21.15.143:0/189369405 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:02:20.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.675+0000 7f578de00640 1 --2- 172.21.15.143:0/189369405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57900af440 0x7f579009db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:20.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.675+0000 7f578e800640 1 --2- 172.21.15.143:0/189369405 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57900cf500 0x7f579015f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:20.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.675+0000 7f578d400640 1 -- 172.21.15.143:0/189369405 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57900cf500 msgr2=0x7f579015f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:20.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.675+0000 7f578d400640 1 --2- 172.21.15.143:0/189369405 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57900cf500 0x7f579015f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:20.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.675+0000 7f578d400640 1 -- 172.21.15.143:0/189369405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57900af440 msgr2=0x7f579009db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:20.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.675+0000 7f578d400640 1 --2- 172.21.15.143:0/189369405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57900af440 0x7f579009db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:20.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.675+0000 7f578d400640 1 -- 172.21.15.143:0/189369405 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5778009b30 con 0x7f57900b1790 2024-01-16T15:02:20.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.675+0000 7f578d400640 1 --2- 172.21.15.143:0/189369405 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57900b1790 0x7f57900b4f60 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5778009e50 tx=0x7f5778004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:20.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.676+0000 7f5786000640 1 -- 172.21.15.143:0/189369405 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5778009510 con 0x7f57900b1790 2024-01-16T15:02:20.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.676+0000 7f5786000640 1 -- 172.21.15.143:0/189369405 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f577803e450 con 0x7f57900b1790 2024-01-16T15:02:20.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.676+0000 7f5786000640 1 -- 172.21.15.143:0/189369405 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577803e800 con 0x7f57900b1790 2024-01-16T15:02:20.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.676+0000 7f5795000640 1 -- 172.21.15.143:0/189369405 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57901e8300 con 0x7f57900b1790 2024-01-16T15:02:20.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.676+0000 7f5795000640 1 -- 172.21.15.143:0/189369405 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f579009e0b0 con 0x7f57900b1790 2024-01-16T15:02:20.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.677+0000 7f5795000640 1 -- 172.21.15.143:0/189369405 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5790067b80 con 0x7f57900b1790 2024-01-16T15:02:20.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.678+0000 7f5786000640 1 -- 172.21.15.143:0/189369405 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7f577803e960 con 0x7f57900b1790 2024-01-16T15:02:20.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.679+0000 7f5786000640 1 --2- 172.21.15.143:0/189369405 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f576806b470 0x7f576806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:20.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.679+0000 7f5786000640 1 -- 172.21.15.143:0/189369405 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57780acd40 con 0x7f57900b1790 2024-01-16T15:02:20.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.679+0000 7f578de00640 1 --2- 172.21.15.143:0/189369405 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f576806b470 0x7f576806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:20.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.679+0000 7f578de00640 1 --2- 172.21.15.143:0/189369405 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f576806b470 0x7f576806d930 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f578000a750 tx=0x7f5780008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:20.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.683+0000 7f5786000640 1 -- 172.21.15.143:0/189369405 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5778077920 con 0x7f57900b1790 2024-01-16T15:02:20.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.903+0000 7f5795000640 1 -- 172.21.15.143:0/189369405 --> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f57900afdb0 con 0x7f576806b470 2024-01-16T15:02:20.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.923+0000 7f5786000640 1 -- 172.21.15.143:0/189369405 <== mgr.4113 v2:172.21.15.143:6832/3153492115 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f578000c520 con 0x7f576806b470 2024-01-16T15:02:20.904 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:02:20.904 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-16T15:02:20.904 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-16T15:02:20.904 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ss1" 2024-01-16T15:02:20.904 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-16T15:02:20.904 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-16T15:02:20.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.926+0000 7f5795000640 1 -- 172.21.15.143:0/189369405 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f576806b470 msgr2=0x7f576806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:20.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.926+0000 7f5795000640 1 --2- 172.21.15.143:0/189369405 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f576806b470 0x7f576806d930 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f578000a750 tx=0x7f5780008040 comp rx=0 tx=0).stop 2024-01-16T15:02:20.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.926+0000 7f5795000640 1 -- 172.21.15.143:0/189369405 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57900b1790 msgr2=0x7f57900b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:20.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.927+0000 7f5795000640 1 --2- 172.21.15.143:0/189369405 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57900b1790 0x7f57900b4f60 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5778009e50 tx=0x7f5778004a40 comp rx=0 tx=0).stop 2024-01-16T15:02:20.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.927+0000 7f5795000640 1 -- 172.21.15.143:0/189369405 shutdown_connections 2024-01-16T15:02:20.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.927+0000 7f5795000640 1 --2- 172.21.15.143:0/189369405 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f576806b470 0x7f576806d930 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:20.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.927+0000 7f5795000640 1 --2- 172.21.15.143:0/189369405 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57900cf500 0x7f579015f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:20.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.927+0000 7f5795000640 1 --2- 172.21.15.143:0/189369405 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57900b1790 0x7f57900b4f60 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:20.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.927+0000 7f5795000640 1 --2- 172.21.15.143:0/189369405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57900af440 0x7f579009db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:20.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.927+0000 7f5795000640 1 -- 172.21.15.143:0/189369405 >> 172.21.15.143:0/189369405 conn(0x7f57900b1cf0 msgr2=0x7f57900b20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:20.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.927+0000 7f5795000640 1 -- 172.21.15.143:0/189369405 shutdown_connections 2024-01-16T15:02:20.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:20.927+0000 7f5795000640 1 -- 172.21.15.143:0/189369405 wait complete. 2024-01-16T15:02:20.927 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-16T15:02:21.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.163+0000 7f5244000640 1 Processor -- start 2024-01-16T15:02:21.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.163+0000 7f5244000640 1 -- start start 2024-01-16T15:02:21.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.165+0000 7f5244000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f523c072160 0x7f523c15e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:21.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.165+0000 7f5244000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f523c0c49f0 0x7f523c06d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:21.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.165+0000 7f5244000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f523c15ec90 con 0x7f523c06dc40 2024-01-16T15:02:21.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.165+0000 7f5244000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f523c06e0f0 con 0x7f523c0c49f0 2024-01-16T15:02:21.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.165+0000 7f5244000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f523c0cd530 con 0x7f523c072160 2024-01-16T15:02:21.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.165+0000 7f5240c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f523c0c49f0 0x7f523c06d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:21.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.165+0000 7f5240c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f523c0c49f0 0x7f523c06d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34402/0 (socket says 172.21.15.143:34402) 2024-01-16T15:02:21.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.165+0000 7f5240c00640 1 -- 172.21.15.143:0/1647073167 learned_addr learned my addr 172.21.15.143:0/1647073167 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:02:21.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.166+0000 7f5241600640 1 --2- 172.21.15.143:0/1647073167 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f523c072160 0x7f523c15e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:21.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.166+0000 7f5240c00640 1 -- 172.21.15.143:0/1647073167 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f523c072160 msgr2=0x7f523c15e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:21.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.166+0000 7f5240c00640 1 --2- 172.21.15.143:0/1647073167 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f523c072160 0x7f523c15e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:21.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.166+0000 7f5240c00640 1 -- 172.21.15.143:0/1647073167 >> v1:172.21.15.204:6789/0 conn(0x7f523c06dc40 legacy=0x7f523c0cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:21.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.166+0000 7f5240c00640 1 -- 172.21.15.143:0/1647073167 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f523c0c4ed0 con 0x7f523c0c49f0 2024-01-16T15:02:21.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.167+0000 7f5240c00640 1 --2- 172.21.15.143:0/1647073167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f523c0c49f0 0x7f523c06d700 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5224009e50 tx=0x7f522402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7263dfbdac5e62db server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:21.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.167+0000 7f523be00640 1 -- 172.21.15.143:0/1647073167 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522402ca70 con 0x7f523c0c49f0 2024-01-16T15:02:21.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.167+0000 7f523be00640 1 -- 172.21.15.143:0/1647073167 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f522402cbd0 con 0x7f523c0c49f0 2024-01-16T15:02:21.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.167+0000 7f523be00640 1 -- 172.21.15.143:0/1647073167 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52240343f0 con 0x7f523c0c49f0 2024-01-16T15:02:21.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.168+0000 7f5244000640 1 -- 172.21.15.143:0/1647073167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f523c0c49f0 msgr2=0x7f523c06d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:21.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.168+0000 7f5244000640 1 --2- 172.21.15.143:0/1647073167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f523c0c49f0 0x7f523c06d700 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5224009e50 tx=0x7f522402bed0 comp rx=0 tx=0).stop 2024-01-16T15:02:21.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.168+0000 7f5244000640 1 -- 172.21.15.143:0/1647073167 shutdown_connections 2024-01-16T15:02:21.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.168+0000 7f5244000640 1 --2- 172.21.15.143:0/1647073167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f523c0c49f0 0x7f523c06d700 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:21.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.168+0000 7f5244000640 1 --2- 172.21.15.143:0/1647073167 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f523c072160 0x7f523c15e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:21.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.168+0000 7f5244000640 1 -- 172.21.15.143:0/1647073167 >> 172.21.15.143:0/1647073167 conn(0x7f523c0b0c50 msgr2=0x7f523c0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:21.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.168+0000 7f5244000640 1 -- 172.21.15.143:0/1647073167 shutdown_connections 2024-01-16T15:02:21.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.168+0000 7f5244000640 1 -- 172.21.15.143:0/1647073167 wait complete. 2024-01-16T15:02:21.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.170+0000 7f5244000640 1 Processor -- start 2024-01-16T15:02:21.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.170+0000 7f5244000640 1 -- start start 2024-01-16T15:02:21.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.171+0000 7f5244000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f523c06dc40 0x7f523c15c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:21.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.171+0000 7f5244000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f523c072160 0x7f523c0b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:21.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.171+0000 7f5244000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f523c0c49f0 0x7f523c0b4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:21.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.171+0000 7f5244000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523c15bd70 con 0x7f523c06dc40 2024-01-16T15:02:21.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.171+0000 7f5244000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523c15ae80 con 0x7f523c072160 2024-01-16T15:02:21.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.171+0000 7f5244000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f523c15b160 con 0x7f523c0c49f0 2024-01-16T15:02:21.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.171+0000 7f5241600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f523c06dc40 0x7f523c15c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:21.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.171+0000 7f5241600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f523c06dc40 0x7f523c15c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34410/0 (socket says 172.21.15.143:34410) 2024-01-16T15:02:21.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.171+0000 7f5241600640 1 -- 172.21.15.143:0/2443541837 learned_addr learned my addr 172.21.15.143:0/2443541837 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:02:21.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.171+0000 7f5240c00640 1 --2- 172.21.15.143:0/2443541837 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f523c072160 0x7f523c0b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:21.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.171+0000 7f5242000640 1 --2- 172.21.15.143:0/2443541837 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f523c0c49f0 0x7f523c0b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:21.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.172+0000 7f5241600640 1 -- 172.21.15.143:0/2443541837 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f523c0c49f0 msgr2=0x7f523c0b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:21.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.172+0000 7f5241600640 1 --2- 172.21.15.143:0/2443541837 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f523c0c49f0 0x7f523c0b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:21.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.172+0000 7f5241600640 1 -- 172.21.15.143:0/2443541837 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f523c072160 msgr2=0x7f523c0b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:21.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.172+0000 7f5241600640 1 --2- 172.21.15.143:0/2443541837 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f523c072160 0x7f523c0b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:21.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.172+0000 7f5241600640 1 -- 172.21.15.143:0/2443541837 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5224009b30 con 0x7f523c06dc40 2024-01-16T15:02:21.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.172+0000 7f5240c00640 1 --2- 172.21.15.143:0/2443541837 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f523c072160 0x7f523c0b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:02:21.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.172+0000 7f5241600640 1 --2- 172.21.15.143:0/2443541837 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f523c06dc40 0x7f523c15c8d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f522c00d800 tx=0x7f522c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:21.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.172+0000 7f5239600640 1 -- 172.21.15.143:0/2443541837 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522c00b6c0 con 0x7f523c06dc40 2024-01-16T15:02:21.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.172+0000 7f5239600640 1 -- 172.21.15.143:0/2443541837 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f522c00b820 con 0x7f523c06dc40 2024-01-16T15:02:21.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.172+0000 7f5239600640 1 -- 172.21.15.143:0/2443541837 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522c00bbb0 con 0x7f523c06dc40 2024-01-16T15:02:21.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.172+0000 7f5244000640 1 -- 172.21.15.143:0/2443541837 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f523c15b440 con 0x7f523c06dc40 2024-01-16T15:02:21.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.172+0000 7f5244000640 1 -- 172.21.15.143:0/2443541837 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f523c15ba00 con 0x7f523c06dc40 2024-01-16T15:02:21.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.173+0000 7f5244000640 1 -- 172.21.15.143:0/2443541837 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f523c061a50 con 0x7f523c06dc40 2024-01-16T15:02:21.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.174+0000 7f5239600640 1 -- 172.21.15.143:0/2443541837 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7f522c00bd10 con 0x7f523c06dc40 2024-01-16T15:02:21.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.176+0000 7f5239600640 1 --2- 172.21.15.143:0/2443541837 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f521406b470 0x7f521406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:21.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.176+0000 7f5239600640 1 -- 172.21.15.143:0/2443541837 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f522c08a520 con 0x7f523c06dc40 2024-01-16T15:02:21.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.176+0000 7f5240c00640 1 --2- 172.21.15.143:0/2443541837 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f521406b470 0x7f521406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:21.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.177+0000 7f5240c00640 1 --2- 172.21.15.143:0/2443541837 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f521406b470 0x7f521406d930 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5224035000 tx=0x7f5224030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:21.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.180+0000 7f5239600640 1 -- 172.21.15.143:0/2443541837 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f522c055070 con 0x7f523c06dc40 2024-01-16T15:02:21.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.402+0000 7f5244000640 1 -- 172.21.15.143:0/2443541837 --> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f523c0c5770 con 0x7f521406b470 2024-01-16T15:02:21.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.444+0000 7f5239600640 1 -- 172.21.15.143:0/2443541837 <== mgr.4113 v2:172.21.15.143:6832/3153492115 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5224040450 con 0x7f521406b470 2024-01-16T15:02:21.424 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:02:21.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.447+0000 7f5244000640 1 -- 172.21.15.143:0/2443541837 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f521406b470 msgr2=0x7f521406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:21.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.447+0000 7f5244000640 1 --2- 172.21.15.143:0/2443541837 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f521406b470 0x7f521406d930 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5224035000 tx=0x7f5224030000 comp rx=0 tx=0).stop 2024-01-16T15:02:21.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.447+0000 7f5244000640 1 -- 172.21.15.143:0/2443541837 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f523c06dc40 msgr2=0x7f523c15c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:21.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.447+0000 7f5244000640 1 --2- 172.21.15.143:0/2443541837 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f523c06dc40 0x7f523c15c8d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f522c00d800 tx=0x7f522c00dcc0 comp rx=0 tx=0).stop 2024-01-16T15:02:21.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.447+0000 7f5244000640 1 -- 172.21.15.143:0/2443541837 shutdown_connections 2024-01-16T15:02:21.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.447+0000 7f5244000640 1 --2- 172.21.15.143:0/2443541837 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f521406b470 0x7f521406d930 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:21.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.447+0000 7f5244000640 1 --2- 172.21.15.143:0/2443541837 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f523c0c49f0 0x7f523c0b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:21.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.447+0000 7f5244000640 1 --2- 172.21.15.143:0/2443541837 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f523c072160 0x7f523c0b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:21.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.447+0000 7f5244000640 1 --2- 172.21.15.143:0/2443541837 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f523c06dc40 0x7f523c15c8d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:21.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.447+0000 7f5244000640 1 -- 172.21.15.143:0/2443541837 >> 172.21.15.143:0/2443541837 conn(0x7f523c0b0c50 msgr2=0x7f523c06d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:21.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.447+0000 7f5244000640 1 -- 172.21.15.143:0/2443541837 shutdown_connections 2024-01-16T15:02:21.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.447+0000 7f5244000640 1 -- 172.21.15.143:0/2443541837 wait complete. 2024-01-16T15:02:21.447 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-01-16T15:02:21.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.673+0000 7fcf5d200640 1 Processor -- start 2024-01-16T15:02:21.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.673+0000 7fcf5d200640 1 -- start start 2024-01-16T15:02:21.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.675+0000 7fcf5d200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcf5815b440 0x7fcf580cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:21.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.675+0000 7fcf5d200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf580cc3a0 con 0x7fcf580cbf20 2024-01-16T15:02:21.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.675+0000 7fcf5d200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf5815eea0 con 0x7fcf580c59b0 2024-01-16T15:02:21.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.675+0000 7fcf5d200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5815b820 con 0x7fcf5815b440 2024-01-16T15:02:21.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.675+0000 7fcf55400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fcf580cbf20 0x7fcf580c52a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49042/0 (socket says 172.21.15.143:49042) 2024-01-16T15:02:21.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.675+0000 7fcf55400640 1 -- 172.21.15.143:0/120654641 learned_addr learned my addr 172.21.15.143:0/120654641 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:02:21.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.675+0000 7fcf55e00640 1 --2- 172.21.15.143:0/120654641 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcf5815b440 0x7fcf580cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:21.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.676+0000 7fcf54a00640 1 -- 172.21.15.143:0/120654641 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1574960520 0 0) 0x7fcf40001110 con 0x7fcf580cbf20 2024-01-16T15:02:21.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.676+0000 7fcf54a00640 1 -- 172.21.15.143:0/120654641 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf38003520 con 0x7fcf580cbf20 2024-01-16T15:02:21.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.676+0000 7fcf54a00640 1 -- 172.21.15.143:0/120654641 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 368334004 0 0) 0x7fcf38003520 con 0x7fcf580cbf20 2024-01-16T15:02:21.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.676+0000 7fcf54a00640 1 -- 172.21.15.143:0/120654641 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcf5815b440 msgr2=0x7fcf580cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:21.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.676+0000 7fcf54a00640 1 --2- 172.21.15.143:0/120654641 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcf5815b440 0x7fcf580cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:21.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.676+0000 7fcf54a00640 1 -- 172.21.15.143:0/120654641 >> v1:172.21.15.204:6789/0 conn(0x7fcf580c59b0 legacy=0x7fcf5806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:21.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.676+0000 7fcf54a00640 1 -- 172.21.15.143:0/120654641 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf580abf30 con 0x7fcf580cbf20 2024-01-16T15:02:21.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.676+0000 7fcf54a00640 1 -- 172.21.15.143:0/120654641 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7fcf40001800 con 0x7fcf580cbf20 2024-01-16T15:02:21.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.676+0000 7fcf54a00640 1 -- 172.21.15.143:0/120654641 >> v1:172.21.15.143:6789/0 conn(0x7fcf580cbf20 legacy=0x7fcf580c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:21.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.676+0000 7fcf54a00640 1 --2- 172.21.15.143:0/120654641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf380039a0 0x7fcf38003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:21.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.676+0000 7fcf54a00640 1 --2- 172.21.15.143:0/120654641 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcf5815b440 0x7fcf380047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:21.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.676+0000 7fcf54a00640 1 --2- 172.21.15.143:0/120654641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf38004d00 0x7fcf38005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:21.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.676+0000 7fcf54a00640 1 -- 172.21.15.143:0/120654641 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5815b820 con 0x7fcf38004d00 2024-01-16T15:02:21.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.676+0000 7fcf54a00640 1 -- 172.21.15.143:0/120654641 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3800bb10 con 0x7fcf380039a0 2024-01-16T15:02:21.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.676+0000 7fcf54a00640 1 -- 172.21.15.143:0/120654641 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3800bcd0 con 0x7fcf5815b440 2024-01-16T15:02:21.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.677+0000 7fcf56800640 1 --2- 172.21.15.143:0/120654641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf38004d00 0x7fcf38005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:21.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.677+0000 7fcf55e00640 1 --2- 172.21.15.143:0/120654641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf380039a0 0x7fcf38003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:21.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.677+0000 7fcf55400640 1 --2- 172.21.15.143:0/120654641 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcf5815b440 0x7fcf380047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:21.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.677+0000 7fcf56800640 1 -- 172.21.15.143:0/120654641 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcf5815b440 msgr2=0x7fcf380047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:21.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.677+0000 7fcf56800640 1 --2- 172.21.15.143:0/120654641 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcf5815b440 0x7fcf380047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:21.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.677+0000 7fcf56800640 1 -- 172.21.15.143:0/120654641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf380039a0 msgr2=0x7fcf38003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:21.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.677+0000 7fcf56800640 1 --2- 172.21.15.143:0/120654641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf380039a0 0x7fcf38003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:21.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.677+0000 7fcf56800640 1 -- 172.21.15.143:0/120654641 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf38003520 con 0x7fcf38004d00 2024-01-16T15:02:21.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.677+0000 7fcf55e00640 1 --2- 172.21.15.143:0/120654641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf380039a0 0x7fcf38003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T15:02:21.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.678+0000 7fcf56800640 1 --2- 172.21.15.143:0/120654641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf38004d00 0x7fcf38005180 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fcf50001e50 tx=0x7fcf380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7df418de51766d8 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:21.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.678+0000 7fcf54a00640 1 -- 172.21.15.143:0/120654641 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf50003b70 con 0x7fcf38004d00 2024-01-16T15:02:21.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.679+0000 7fcf54a00640 1 -- 172.21.15.143:0/120654641 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf50003cd0 con 0x7fcf38004d00 2024-01-16T15:02:21.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.679+0000 7fcf54a00640 1 -- 172.21.15.143:0/120654641 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf50005310 con 0x7fcf38004d00 2024-01-16T15:02:21.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.679+0000 7fcf5d200640 1 -- 172.21.15.143:0/120654641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf38004d00 msgr2=0x7fcf38005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:21.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.679+0000 7fcf5d200640 1 --2- 172.21.15.143:0/120654641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf38004d00 0x7fcf38005180 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fcf50001e50 tx=0x7fcf380057f0 comp rx=0 tx=0).stop 2024-01-16T15:02:21.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.679+0000 7fcf56800640 1 -- 172.21.15.143:0/120654641 reap_dead start 2024-01-16T15:02:21.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.679+0000 7fcf5d200640 1 -- 172.21.15.143:0/120654641 shutdown_connections 2024-01-16T15:02:21.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.679+0000 7fcf5d200640 1 -- 172.21.15.143:0/120654641 >> 172.21.15.143:0/120654641 conn(0x7fcf580b0c50 msgr2=0x7fcf580b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:21.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.679+0000 7fcf5d200640 1 -- 172.21.15.143:0/120654641 shutdown_connections 2024-01-16T15:02:21.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.679+0000 7fcf5d200640 1 -- 172.21.15.143:0/120654641 wait complete. 2024-01-16T15:02:21.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.681+0000 7fcf5d200640 1 Processor -- start 2024-01-16T15:02:21.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.681+0000 7fcf5d200640 1 -- start start 2024-01-16T15:02:21.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.682+0000 7fcf5d200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf5814dc70 0x7fcf581563e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:21.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.682+0000 7fcf5d200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf58156920 0x7fcf5809bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:21.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.682+0000 7fcf5d200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcf5809c1a0 0x7fcf580999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:21.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.682+0000 7fcf5d200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5815c680 con 0x7fcf58156920 2024-01-16T15:02:21.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.682+0000 7fcf5d200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5814e050 con 0x7fcf5814dc70 2024-01-16T15:02:21.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.682+0000 7fcf5d200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf58156d80 con 0x7fcf5809c1a0 2024-01-16T15:02:21.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.682+0000 7fcf55400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf58156920 0x7fcf5809bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:21.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.683+0000 7fcf55400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf58156920 0x7fcf5809bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34434/0 (socket says 172.21.15.143:34434) 2024-01-16T15:02:21.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.683+0000 7fcf55400640 1 -- 172.21.15.143:0/234300046 learned_addr learned my addr 172.21.15.143:0/234300046 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:02:21.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.683+0000 7fcf56800640 1 --2- 172.21.15.143:0/234300046 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcf5809c1a0 0x7fcf580999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:21.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.683+0000 7fcf55e00640 1 --2- 172.21.15.143:0/234300046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf5814dc70 0x7fcf581563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:21.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.683+0000 7fcf55400640 1 -- 172.21.15.143:0/234300046 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcf5809c1a0 msgr2=0x7fcf580999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:21.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.683+0000 7fcf55400640 1 --2- 172.21.15.143:0/234300046 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcf5809c1a0 0x7fcf580999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:21.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.683+0000 7fcf55400640 1 -- 172.21.15.143:0/234300046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf5814dc70 msgr2=0x7fcf581563e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:21.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.683+0000 7fcf55400640 1 --2- 172.21.15.143:0/234300046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf5814dc70 0x7fcf581563e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:21.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.683+0000 7fcf55400640 1 -- 172.21.15.143:0/234300046 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf5815ba00 con 0x7fcf58156920 2024-01-16T15:02:21.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.683+0000 7fcf55e00640 1 --2- 172.21.15.143:0/234300046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf5814dc70 0x7fcf581563e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:02:21.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.683+0000 7fcf55400640 1 --2- 172.21.15.143:0/234300046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf58156920 0x7fcf5809bc60 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fcf4000e7e0 tx=0x7fcf4000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:21.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.683+0000 7fcf4e000640 1 -- 172.21.15.143:0/234300046 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4000c8d0 con 0x7fcf58156920 2024-01-16T15:02:21.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.684+0000 7fcf4e000640 1 -- 172.21.15.143:0/234300046 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf4000ca30 con 0x7fcf58156920 2024-01-16T15:02:21.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.684+0000 7fcf5d200640 1 -- 172.21.15.143:0/234300046 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf5815bce0 con 0x7fcf58156920 2024-01-16T15:02:21.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.684+0000 7fcf4e000640 1 -- 172.21.15.143:0/234300046 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4000cde0 con 0x7fcf58156920 2024-01-16T15:02:21.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.684+0000 7fcf5d200640 1 -- 172.21.15.143:0/234300046 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf5815c2a0 con 0x7fcf58156920 2024-01-16T15:02:21.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.685+0000 7fcf5d200640 1 -- 172.21.15.143:0/234300046 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf5815c460 con 0x7fcf58156920 2024-01-16T15:02:21.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.686+0000 7fcf4e000640 1 -- 172.21.15.143:0/234300046 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7fcf40013420 con 0x7fcf58156920 2024-01-16T15:02:21.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.686+0000 7fcf4e000640 1 --2- 172.21.15.143:0/234300046 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fcf3006b4a0 0x7fcf3006d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:21.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.687+0000 7fcf4e000640 1 -- 172.21.15.143:0/234300046 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf40089b40 con 0x7fcf58156920 2024-01-16T15:02:21.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.687+0000 7fcf55e00640 1 --2- 172.21.15.143:0/234300046 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fcf3006b4a0 0x7fcf3006d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:21.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.687+0000 7fcf55e00640 1 --2- 172.21.15.143:0/234300046 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fcf3006b4a0 0x7fcf3006d960 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcf48009a00 tx=0x7fcf48008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:21.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.691+0000 7fcf4e000640 1 -- 172.21.15.143:0/234300046 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcf40054690 con 0x7fcf58156920 2024-01-16T15:02:21.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.914+0000 7fcf5d200640 1 -- 172.21.15.143:0/234300046 --> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf5815f6b0 con 0x7fcf3006b4a0 2024-01-16T15:02:21.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.942+0000 7fcf4e000640 1 -- 172.21.15.143:0/234300046 <== mgr.4113 v2:172.21.15.143:6832/3153492115 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fcf48012950 con 0x7fcf3006b4a0 2024-01-16T15:02:21.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.945+0000 7fcf5d200640 1 -- 172.21.15.143:0/234300046 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fcf3006b4a0 msgr2=0x7fcf3006d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:21.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.945+0000 7fcf5d200640 1 --2- 172.21.15.143:0/234300046 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fcf3006b4a0 0x7fcf3006d960 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcf48009a00 tx=0x7fcf48008040 comp rx=0 tx=0).stop 2024-01-16T15:02:21.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.945+0000 7fcf5d200640 1 -- 172.21.15.143:0/234300046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf58156920 msgr2=0x7fcf5809bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:21.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.945+0000 7fcf5d200640 1 --2- 172.21.15.143:0/234300046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf58156920 0x7fcf5809bc60 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fcf4000e7e0 tx=0x7fcf4000eca0 comp rx=0 tx=0).stop 2024-01-16T15:02:21.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.945+0000 7fcf5d200640 1 -- 172.21.15.143:0/234300046 shutdown_connections 2024-01-16T15:02:21.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.945+0000 7fcf5d200640 1 --2- 172.21.15.143:0/234300046 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fcf3006b4a0 0x7fcf3006d960 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:21.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.945+0000 7fcf5d200640 1 --2- 172.21.15.143:0/234300046 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcf5809c1a0 0x7fcf580999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:21.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.945+0000 7fcf5d200640 1 --2- 172.21.15.143:0/234300046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf58156920 0x7fcf5809bc60 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:21.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.945+0000 7fcf5d200640 1 --2- 172.21.15.143:0/234300046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf5814dc70 0x7fcf581563e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:21.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.945+0000 7fcf5d200640 1 -- 172.21.15.143:0/234300046 >> 172.21.15.143:0/234300046 conn(0x7fcf580b0c50 msgr2=0x7fcf580ae970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:21.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.945+0000 7fcf5d200640 1 -- 172.21.15.143:0/234300046 shutdown_connections 2024-01-16T15:02:21.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:21.945+0000 7fcf5d200640 1 -- 172.21.15.143:0/234300046 wait complete. 2024-01-16T15:02:21.928 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T15:02:21.948+0000 7f2772800640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T15:02:21.929 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T15:02:21.948+0000 7f2772800640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T15:02:21.929 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T15:02:21.948+0000 7f2772800640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T15:02:21.929 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T15:02:21.948+0000 7f2772800640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T15:02:21.929 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T15:02:21.948+0000 7f2772800640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T15:02:21.946 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-01-16T15:02:21.947 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T15:02:21.967+0000 7f2773c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T15:02:21.947 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T15:02:21.967+0000 7f2773c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T15:02:21.947 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T15:02:21.967+0000 7f2773c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T15:02:21.947 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T15:02:21.967+0000 7f2773c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T15:02:21.948 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T15:02:21.967+0000 7f2773c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T15:02:22.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.153+0000 7fcdc7200640 1 Processor -- start 2024-01-16T15:02:22.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.153+0000 7fcdc7200640 1 -- start start 2024-01-16T15:02:22.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.154+0000 7fcdc7200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcdc815a1d0 0x7fcdc80b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:22.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.154+0000 7fcdc7200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdc815a5b0 con 0x7fcdc80cd060 2024-01-16T15:02:22.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.154+0000 7fcdc7200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdc80ce3f0 con 0x7fcdc80ceb20 2024-01-16T15:02:22.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.154+0000 7fcdc7200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdc80b3f70 con 0x7fcdc815a1d0 2024-01-16T15:02:22.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.155+0000 7fcdc5e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcdc815a1d0 0x7fcdc80b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:22.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.155+0000 7fcdc5e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcdc815a1d0 0x7fcdc80b3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34482/0 (socket says 172.21.15.143:34482) 2024-01-16T15:02:22.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.155+0000 7fcdc5e00640 1 -- 172.21.15.143:0/1366548715 learned_addr learned my addr 172.21.15.143:0/1366548715 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:02:22.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.155+0000 7fcdc5e00640 1 -- 172.21.15.143:0/1366548715 >> v1:172.21.15.204:6789/0 conn(0x7fcdc80ceb20 legacy=0x7fcdc80cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:22.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.155+0000 7fcdc5e00640 1 -- 172.21.15.143:0/1366548715 >> v1:172.21.15.143:6789/0 conn(0x7fcdc80cd060 legacy=0x7fcdc815de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:22.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.155+0000 7fcdc5e00640 1 -- 172.21.15.143:0/1366548715 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc80cd7e0 con 0x7fcdc815a1d0 2024-01-16T15:02:22.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.156+0000 7fcdc5e00640 1 --2- 172.21.15.143:0/1366548715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcdc815a1d0 0x7fcdc80b3a30 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0009e50 tx=0x7fcdc002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342b956de211349c server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:22.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.157+0000 7fcdc4a00640 1 -- 172.21.15.143:0/1366548715 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc002cb00 con 0x7fcdc815a1d0 2024-01-16T15:02:22.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.157+0000 7fcdc4a00640 1 -- 172.21.15.143:0/1366548715 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdc002cc60 con 0x7fcdc815a1d0 2024-01-16T15:02:22.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.157+0000 7fcdc4a00640 1 -- 172.21.15.143:0/1366548715 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc00343f0 con 0x7fcdc815a1d0 2024-01-16T15:02:22.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.157+0000 7fcdc7200640 1 -- 172.21.15.143:0/1366548715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcdc815a1d0 msgr2=0x7fcdc80b3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:22.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.157+0000 7fcdc7200640 1 --2- 172.21.15.143:0/1366548715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcdc815a1d0 0x7fcdc80b3a30 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0009e50 tx=0x7fcdc002bf60 comp rx=0 tx=0).stop 2024-01-16T15:02:22.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.157+0000 7fcdc7200640 1 -- 172.21.15.143:0/1366548715 shutdown_connections 2024-01-16T15:02:22.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.157+0000 7fcdc7200640 1 --2- 172.21.15.143:0/1366548715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcdc815a1d0 0x7fcdc80b3a30 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:22.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.158+0000 7fcdc7200640 1 -- 172.21.15.143:0/1366548715 >> 172.21.15.143:0/1366548715 conn(0x7fcdc80b1ce0 msgr2=0x7fcdc80cc240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:22.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.158+0000 7fcdc7200640 1 -- 172.21.15.143:0/1366548715 shutdown_connections 2024-01-16T15:02:22.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.158+0000 7fcdc7200640 1 -- 172.21.15.143:0/1366548715 wait complete. 2024-01-16T15:02:22.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.160+0000 7fcdc7200640 1 Processor -- start 2024-01-16T15:02:22.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.160+0000 7fcdc7200640 1 -- start start 2024-01-16T15:02:22.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.160+0000 7fcdc7200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcdc80cd060 0x7fcdc8144570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:22.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.160+0000 7fcdc7200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdc80ceb20 0x7fcdc80a9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:22.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.160+0000 7fcdc7200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcdc815a1d0 0x7fcdc815c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:22.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.160+0000 7fcdc7200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc8146a70 con 0x7fcdc815a1d0 2024-01-16T15:02:22.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.160+0000 7fcdc7200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc8146d50 con 0x7fcdc80ceb20 2024-01-16T15:02:22.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.160+0000 7fcdc7200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc8147030 con 0x7fcdc80cd060 2024-01-16T15:02:22.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.161+0000 7fcdc6800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcdc815a1d0 0x7fcdc815c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:22.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.161+0000 7fcdc6800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcdc815a1d0 0x7fcdc815c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34490/0 (socket says 172.21.15.143:34490) 2024-01-16T15:02:22.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.161+0000 7fcdc6800640 1 -- 172.21.15.143:0/1318909939 learned_addr learned my addr 172.21.15.143:0/1318909939 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:02:22.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.161+0000 7fcdc5400640 1 --2- 172.21.15.143:0/1318909939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdc80ceb20 0x7fcdc80a9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:22.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.161+0000 7fcdc5e00640 1 --2- 172.21.15.143:0/1318909939 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcdc80cd060 0x7fcdc8144570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:22.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.161+0000 7fcdc6800640 1 -- 172.21.15.143:0/1318909939 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcdc80cd060 msgr2=0x7fcdc8144570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:22.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.161+0000 7fcdc6800640 1 --2- 172.21.15.143:0/1318909939 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcdc80cd060 0x7fcdc8144570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:22.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.161+0000 7fcdc6800640 1 -- 172.21.15.143:0/1318909939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdc80ceb20 msgr2=0x7fcdc80a9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:22.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.161+0000 7fcdc6800640 1 --2- 172.21.15.143:0/1318909939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdc80ceb20 0x7fcdc80a9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:22.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.161+0000 7fcdc6800640 1 -- 172.21.15.143:0/1318909939 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc0009b30 con 0x7fcdc815a1d0 2024-01-16T15:02:22.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.161+0000 7fcdc5400640 1 --2- 172.21.15.143:0/1318909939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdc80ceb20 0x7fcdc80a9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:02:22.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.162+0000 7fcdc6800640 1 --2- 172.21.15.143:0/1318909939 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcdc815a1d0 0x7fcdc815c0c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fcdb800c3a0 tx=0x7fcdb800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:22.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.162+0000 7fcdbe000640 1 -- 172.21.15.143:0/1318909939 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb80052d0 con 0x7fcdc815a1d0 2024-01-16T15:02:22.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.162+0000 7fcdbe000640 1 -- 172.21.15.143:0/1318909939 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb8005430 con 0x7fcdc815a1d0 2024-01-16T15:02:22.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.162+0000 7fcdbe000640 1 -- 172.21.15.143:0/1318909939 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8003830 con 0x7fcdc815a1d0 2024-01-16T15:02:22.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.162+0000 7fcdc7200640 1 -- 172.21.15.143:0/1318909939 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdc8147310 con 0x7fcdc815a1d0 2024-01-16T15:02:22.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.162+0000 7fcdc7200640 1 -- 172.21.15.143:0/1318909939 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc80a9aa0 con 0x7fcdc815a1d0 2024-01-16T15:02:22.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.163+0000 7fcdc7200640 1 -- 172.21.15.143:0/1318909939 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdc804e840 con 0x7fcdc815a1d0 2024-01-16T15:02:22.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.164+0000 7fcdbe000640 1 -- 172.21.15.143:0/1318909939 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7fcdb8003990 con 0x7fcdc815a1d0 2024-01-16T15:02:22.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.165+0000 7fcdbe000640 1 --2- 172.21.15.143:0/1318909939 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fcda006b470 0x7fcda006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:22.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.165+0000 7fcdbe000640 1 -- 172.21.15.143:0/1318909939 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcdb8089470 con 0x7fcdc815a1d0 2024-01-16T15:02:22.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.165+0000 7fcdc5e00640 1 --2- 172.21.15.143:0/1318909939 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fcda006b470 0x7fcda006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:22.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.166+0000 7fcdc5e00640 1 --2- 172.21.15.143:0/1318909939 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fcda006b470 0x7fcda006d930 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0035000 tx=0x7fcdc0030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:22.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.174+0000 7fcdbe000640 1 -- 172.21.15.143:0/1318909939 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcdb8054110 con 0x7fcdc815a1d0 2024-01-16T15:02:22.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.391+0000 7fcdc7200640 1 -- 172.21.15.143:0/1318909939 --> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7fcdc80ce080 con 0x7fcda006b470 2024-01-16T15:02:22.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.404+0000 7fcdbe000640 1 -- 172.21.15.143:0/1318909939 <== mgr.4113 v2:172.21.15.143:6832/3153492115 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fcdc00440c0 con 0x7fcda006b470 2024-01-16T15:02:22.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.406+0000 7fcdc7200640 1 -- 172.21.15.143:0/1318909939 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fcda006b470 msgr2=0x7fcda006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:22.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.406+0000 7fcdc7200640 1 --2- 172.21.15.143:0/1318909939 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fcda006b470 0x7fcda006d930 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0035000 tx=0x7fcdc0030000 comp rx=0 tx=0).stop 2024-01-16T15:02:22.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.406+0000 7fcdc7200640 1 -- 172.21.15.143:0/1318909939 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcdc815a1d0 msgr2=0x7fcdc815c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:22.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.406+0000 7fcdc7200640 1 --2- 172.21.15.143:0/1318909939 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcdc815a1d0 0x7fcdc815c0c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fcdb800c3a0 tx=0x7fcdb800c860 comp rx=0 tx=0).stop 2024-01-16T15:02:22.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.406+0000 7fcdc7200640 1 -- 172.21.15.143:0/1318909939 shutdown_connections 2024-01-16T15:02:22.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.406+0000 7fcdc7200640 1 --2- 172.21.15.143:0/1318909939 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fcda006b470 0x7fcda006d930 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:22.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.406+0000 7fcdc7200640 1 --2- 172.21.15.143:0/1318909939 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcdc815a1d0 0x7fcdc815c0c0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:22.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.407+0000 7fcdc7200640 1 --2- 172.21.15.143:0/1318909939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdc80ceb20 0x7fcdc80a9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:22.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.407+0000 7fcdc7200640 1 --2- 172.21.15.143:0/1318909939 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcdc80cd060 0x7fcdc8144570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:22.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.407+0000 7fcdc7200640 1 -- 172.21.15.143:0/1318909939 >> 172.21.15.143:0/1318909939 conn(0x7fcdc80b1ce0 msgr2=0x7fcdc80b0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:22.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.407+0000 7fcdc7200640 1 -- 172.21.15.143:0/1318909939 shutdown_connections 2024-01-16T15:02:22.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:22.407+0000 7fcdc7200640 1 -- 172.21.15.143:0/1318909939 wait complete. 2024-01-16T15:02:22.398 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T15:02:22.418+0000 7f2773200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T15:02:22.398 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T15:02:22.418+0000 7f2773200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T15:02:22.398 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T15:02:22.418+0000 7f2773200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T15:02:22.398 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T15:02:22.418+0000 7f2773200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T15:02:22.398 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-16T15:02:22.418+0000 7f2773200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T15:02:22.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:02:22.409 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-01-16T15:02:22.457 INFO:teuthology.orchestra.run.smithi143.stdout:3 2024-01-16T15:02:23.586 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:02:23.594 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:02:27.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:02:27.459 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-01-16T15:02:27.519 INFO:teuthology.orchestra.run.smithi143.stdout:2 2024-01-16T15:02:27.520 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs --yes-i-really-mean-it 2024-01-16T15:02:27.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.757+0000 7feff0600640 1 Processor -- start 2024-01-16T15:02:27.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.757+0000 7feff0600640 1 -- start start 2024-01-16T15:02:27.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.759+0000 7feff0600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fefe80c59b0 0x7fefe8166230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:27.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.759+0000 7feff0600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefe80c5dc0 con 0x7fefe815b440 2024-01-16T15:02:27.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.759+0000 7feff0600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefe815eea0 con 0x7fefe80cc0f0 2024-01-16T15:02:27.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.759+0000 7feff0600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefe815b820 con 0x7fefe80c59b0 2024-01-16T15:02:27.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.759+0000 7fefee600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fefe80c59b0 0x7fefe8166230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:27.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.759+0000 7fefee600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fefe80c59b0 0x7fefe8166230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57938/0 (socket says 172.21.15.143:57938) 2024-01-16T15:02:27.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.759+0000 7fefedc00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fefe815b440 0x7fefe80cb9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33950/0 (socket says 172.21.15.143:33950) 2024-01-16T15:02:27.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.759+0000 7fefee600640 1 -- 172.21.15.143:0/2359647953 learned_addr learned my addr 172.21.15.143:0/2359647953 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:02:27.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.760+0000 7fefee600640 1 -- 172.21.15.143:0/2359647953 >> v1:172.21.15.204:6789/0 conn(0x7fefe80cc0f0 legacy=0x7fefe80c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:27.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.760+0000 7fefee600640 1 -- 172.21.15.143:0/2359647953 >> v1:172.21.15.143:6789/0 conn(0x7fefe815b440 legacy=0x7fefe80cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:27.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.760+0000 7fefee600640 1 -- 172.21.15.143:0/2359647953 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe806d600 con 0x7fefe80c59b0 2024-01-16T15:02:27.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.761+0000 7fefee600640 1 --2- 172.21.15.143:0/2359647953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fefe80c59b0 0x7fefe8166230 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fefe0009870 tx=0x7fefe002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7cc3b89238b0d7 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:27.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.761+0000 7fefe7e00640 1 -- 172.21.15.143:0/2359647953 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe0031860 con 0x7fefe80c59b0 2024-01-16T15:02:27.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.761+0000 7fefe7e00640 1 -- 172.21.15.143:0/2359647953 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe00319c0 con 0x7fefe80c59b0 2024-01-16T15:02:27.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.761+0000 7fefe7e00640 1 -- 172.21.15.143:0/2359647953 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe0031d50 con 0x7fefe80c59b0 2024-01-16T15:02:27.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.762+0000 7feff0600640 1 -- 172.21.15.143:0/2359647953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fefe80c59b0 msgr2=0x7fefe8166230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:27.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.762+0000 7feff0600640 1 --2- 172.21.15.143:0/2359647953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fefe80c59b0 0x7fefe8166230 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fefe0009870 tx=0x7fefe002b900 comp rx=0 tx=0).stop 2024-01-16T15:02:27.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.762+0000 7feff0600640 1 -- 172.21.15.143:0/2359647953 shutdown_connections 2024-01-16T15:02:27.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.762+0000 7feff0600640 1 --2- 172.21.15.143:0/2359647953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fefe80c59b0 0x7fefe8166230 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:27.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.762+0000 7feff0600640 1 -- 172.21.15.143:0/2359647953 >> 172.21.15.143:0/2359647953 conn(0x7fefe80b0c50 msgr2=0x7fefe80b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:27.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.762+0000 7feff0600640 1 -- 172.21.15.143:0/2359647953 shutdown_connections 2024-01-16T15:02:27.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.762+0000 7feff0600640 1 -- 172.21.15.143:0/2359647953 wait complete. 2024-01-16T15:02:27.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.764+0000 7feff0600640 1 Processor -- start 2024-01-16T15:02:27.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.764+0000 7feff0600640 1 -- start start 2024-01-16T15:02:27.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.765+0000 7feff0600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fefe80c59b0 0x7fefe8153ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:27.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.765+0000 7feff0600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fefe80cc0f0 0x7fefe80b2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:27.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.765+0000 7feff0600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefe815b440 0x7fefe80b3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:27.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.765+0000 7feff0600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe80b4360 con 0x7fefe80cc0f0 2024-01-16T15:02:27.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.765+0000 7feff0600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe80b37e0 con 0x7fefe815b440 2024-01-16T15:02:27.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.765+0000 7feff0600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe80b3ac0 con 0x7fefe80c59b0 2024-01-16T15:02:27.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.765+0000 7fefed200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fefe80cc0f0 0x7fefe80b2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:27.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.765+0000 7fefed200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fefe80cc0f0 0x7fefe80b2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57944/0 (socket says 172.21.15.143:57944) 2024-01-16T15:02:27.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.765+0000 7fefed200640 1 -- 172.21.15.143:0/1260118517 learned_addr learned my addr 172.21.15.143:0/1260118517 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:02:27.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.765+0000 7fefedc00640 1 --2- 172.21.15.143:0/1260118517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fefe80c59b0 0x7fefe8153ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:27.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.766+0000 7fefee600640 1 --2- 172.21.15.143:0/1260118517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefe815b440 0x7fefe80b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:27.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.766+0000 7fefed200640 1 -- 172.21.15.143:0/1260118517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fefe80c59b0 msgr2=0x7fefe8153ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:27.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.766+0000 7fefed200640 1 --2- 172.21.15.143:0/1260118517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fefe80c59b0 0x7fefe8153ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:27.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.766+0000 7fefed200640 1 -- 172.21.15.143:0/1260118517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefe815b440 msgr2=0x7fefe80b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:27.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.766+0000 7fefed200640 1 --2- 172.21.15.143:0/1260118517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefe815b440 0x7fefe80b3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:27.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.766+0000 7fefed200640 1 -- 172.21.15.143:0/1260118517 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe00094d0 con 0x7fefe80cc0f0 2024-01-16T15:02:27.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.766+0000 7fefedc00640 1 --2- 172.21.15.143:0/1260118517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fefe80c59b0 0x7fefe8153ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:02:27.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.766+0000 7fefee600640 1 --2- 172.21.15.143:0/1260118517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefe815b440 0x7fefe80b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:02:27.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.766+0000 7fefed200640 1 --2- 172.21.15.143:0/1260118517 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fefe80cc0f0 0x7fefe80b2c20 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fefd000c7c0 tx=0x7fefd000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:27.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.766+0000 7fefe5600640 1 -- 172.21.15.143:0/1260118517 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd000aa50 con 0x7fefe80cc0f0 2024-01-16T15:02:27.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.766+0000 7fefe5600640 1 -- 172.21.15.143:0/1260118517 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd000abb0 con 0x7fefe80cc0f0 2024-01-16T15:02:27.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.766+0000 7fefe5600640 1 -- 172.21.15.143:0/1260118517 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd0012420 con 0x7fefe80cc0f0 2024-01-16T15:02:27.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.766+0000 7feff0600640 1 -- 172.21.15.143:0/1260118517 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefe80b3da0 con 0x7fefe80cc0f0 2024-01-16T15:02:27.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.766+0000 7feff0600640 1 -- 172.21.15.143:0/1260118517 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefe8159990 con 0x7fefe80cc0f0 2024-01-16T15:02:27.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.768+0000 7feff0600640 1 -- 172.21.15.143:0/1260118517 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefe804e8b0 con 0x7fefe80cc0f0 2024-01-16T15:02:27.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.768+0000 7fefe5600640 1 -- 172.21.15.143:0/1260118517 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 8) v1 ==== 89845+0+0 (secure 0 0 0) 0x7fefd0012580 con 0x7fefe80cc0f0 2024-01-16T15:02:27.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.769+0000 7fefe5600640 1 --2- 172.21.15.143:0/1260118517 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fefc006b620 0x7fefc006dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:27.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.770+0000 7fefe5600640 1 -- 172.21.15.143:0/1260118517 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fefd008a380 con 0x7fefe80cc0f0 2024-01-16T15:02:27.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.770+0000 7fefedc00640 1 --2- 172.21.15.143:0/1260118517 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fefc006b620 0x7fefc006dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:27.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.770+0000 7fefedc00640 1 --2- 172.21.15.143:0/1260118517 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fefc006b620 0x7fefc006dae0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fefd8000f60 tx=0x7fefd8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:27.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.774+0000 7fefe5600640 1 -- 172.21.15.143:0/1260118517 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fefd0054da0 con 0x7fefe80cc0f0 2024-01-16T15:02:27.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:27.991+0000 7feff0600640 1 -- 172.21.15.143:0/1260118517 --> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "cephfs", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7fefe80b1030 con 0x7fefc006b620 2024-01-16T15:02:28.004 INFO:tasks.ceph.mon.a.smithi143.stderr:2024-01-16T15:02:28.025+0000 7f2127a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-16T15:02:28.105 INFO:tasks.ceph.mds.a.smithi143.stdout:starting mds.a at 2024-01-16T15:02:30.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:30.074+0000 7fefe5600640 1 -- 172.21.15.143:0/1260118517 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mgrmap(e 9) v1 ==== 89794+0+0 (secure 0 0 0) 0x7fefd0055170 con 0x7fefe80cc0f0 2024-01-16T15:02:31.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.052+0000 7fefe5600640 1 -- 172.21.15.143:0/1260118517 <== mgr.4113 v2:172.21.15.143:6832/3153492115 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+66 (secure 0 0 0) 0x7fefe80b1030 con 0x7fefc006b620 2024-01-16T15:02:31.032 INFO:teuthology.orchestra.run.smithi143.stdout:metadata pool: cephfs_metadata data pool: ['cephfs_data'] removed 2024-01-16T15:02:31.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.057+0000 7feff0600640 1 -- 172.21.15.143:0/1260118517 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fefc006b620 msgr2=0x7fefc006dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:31.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.057+0000 7feff0600640 1 --2- 172.21.15.143:0/1260118517 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fefc006b620 0x7fefc006dae0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fefd8000f60 tx=0x7fefd8009040 comp rx=0 tx=0).stop 2024-01-16T15:02:31.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.057+0000 7feff0600640 1 -- 172.21.15.143:0/1260118517 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fefe80cc0f0 msgr2=0x7fefe80b2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:31.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.057+0000 7feff0600640 1 --2- 172.21.15.143:0/1260118517 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fefe80cc0f0 0x7fefe80b2c20 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fefd000c7c0 tx=0x7fefd000cc80 comp rx=0 tx=0).stop 2024-01-16T15:02:31.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.057+0000 7feff0600640 1 -- 172.21.15.143:0/1260118517 shutdown_connections 2024-01-16T15:02:31.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.057+0000 7feff0600640 1 --2- 172.21.15.143:0/1260118517 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fefc006b620 0x7fefc006dae0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:31.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.057+0000 7feff0600640 1 --2- 172.21.15.143:0/1260118517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefe815b440 0x7fefe80b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:31.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.057+0000 7feff0600640 1 --2- 172.21.15.143:0/1260118517 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fefe80cc0f0 0x7fefe80b2c20 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:31.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.057+0000 7feff0600640 1 --2- 172.21.15.143:0/1260118517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fefe80c59b0 0x7fefe8153ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:31.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.057+0000 7feff0600640 1 -- 172.21.15.143:0/1260118517 >> 172.21.15.143:0/1260118517 conn(0x7fefe80b0c50 msgr2=0x7fefe80cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:31.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.057+0000 7feff0600640 1 -- 172.21.15.143:0/1260118517 shutdown_connections 2024-01-16T15:02:31.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.057+0000 7feff0600640 1 -- 172.21.15.143:0/1260118517 wait complete. 2024-01-16T15:02:31.063 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-16T15:02:31.063 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-16T15:02:31.193 DEBUG:teuthology.orchestra.run.smithi204:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-16T15:02:31.293 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-16T15:02:31.293 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T15:02:31.293 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T15:02:31.348 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-16T15:02:31.349 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T15:02:31.400 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T15:02:31.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.630+0000 7f9532800640 1 Processor -- start 2024-01-16T15:02:31.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.630+0000 7f9532800640 1 -- start start 2024-01-16T15:02:31.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.631+0000 7f9532800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f952c0cf450 0x7f952c0cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:31.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.631+0000 7f9532800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f952c0cece0 con 0x7f952c15c7f0 2024-01-16T15:02:31.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.631+0000 7f9532800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f952c06d040 con 0x7f952c0cdfb0 2024-01-16T15:02:31.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.631+0000 7f9532800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f952c0cfe40 con 0x7f952c0cf450 2024-01-16T15:02:31.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.632+0000 7f952b400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f952c15c7f0 0x7f952c06db20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33954/0 (socket says 172.21.15.143:33954) 2024-01-16T15:02:31.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.632+0000 7f952b400640 1 -- 172.21.15.143:0/3092072909 learned_addr learned my addr 172.21.15.143:0/3092072909 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:02:31.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.632+0000 7f952a000640 1 -- 172.21.15.143:0/3092072909 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1320406879 0 0) 0x7f952c0cece0 con 0x7f952c15c7f0 2024-01-16T15:02:31.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.632+0000 7f952a000640 1 -- 172.21.15.143:0/3092072909 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f950c003520 con 0x7f952c15c7f0 2024-01-16T15:02:31.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.632+0000 7f952aa00640 1 --2- 172.21.15.143:0/3092072909 >> v2:172.21.15.204:3300/0 conn(0x7f952c0cf450 0x7f952c0cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:31.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.632+0000 7f952a000640 1 -- 172.21.15.143:0/3092072909 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3060354372 0 0) 0x7f950c003520 con 0x7f952c15c7f0 2024-01-16T15:02:31.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.632+0000 7f952a000640 1 -- 172.21.15.143:0/3092072909 >> v2:172.21.15.204:3300/0 conn(0x7f952c0cf450 msgr2=0x7f952c0cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:31.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.632+0000 7f952a000640 1 --2- 172.21.15.143:0/3092072909 >> v2:172.21.15.204:3300/0 conn(0x7f952c0cf450 0x7f952c0cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:31.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.632+0000 7f952a000640 1 -- 172.21.15.143:0/3092072909 >> v1:172.21.15.204:6789/0 conn(0x7f952c0cdfb0 legacy=0x7f952c0ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:31.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.633+0000 7f952a000640 1 -- 172.21.15.143:0/3092072909 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952c06d320 con 0x7f952c15c7f0 2024-01-16T15:02:31.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.633+0000 7f952a000640 1 -- 172.21.15.143:0/3092072909 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2466823303 0 0) 0x7f951c001670 con 0x7f952c15c7f0 2024-01-16T15:02:31.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.633+0000 7f952a000640 1 -- 172.21.15.143:0/3092072909 >> v1:172.21.15.143:6789/0 conn(0x7f952c15c7f0 legacy=0x7f952c06db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:31.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.633+0000 7f952a000640 1 --2- 172.21.15.143:0/3092072909 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f950c003970 0x7f950c003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:31.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.633+0000 7f952a000640 1 --2- 172.21.15.143:0/3092072909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f950c0047c0 0x7f950c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:31.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.633+0000 7f952a000640 1 --2- 172.21.15.143:0/3092072909 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f950c005180 0x7f950c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:31.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.633+0000 7f952a000640 1 -- 172.21.15.143:0/3092072909 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952c0cfe40 con 0x7f950c003970 2024-01-16T15:02:31.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.633+0000 7f952a000640 1 -- 172.21.15.143:0/3092072909 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f950c00c0b0 con 0x7f950c0047c0 2024-01-16T15:02:31.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.633+0000 7f952a000640 1 -- 172.21.15.143:0/3092072909 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f950c00c270 con 0x7f950c005180 2024-01-16T15:02:31.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.633+0000 7f952b400640 1 --2- 172.21.15.143:0/3092072909 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f950c003970 0x7f950c003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:31.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.633+0000 7f952aa00640 1 --2- 172.21.15.143:0/3092072909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f950c0047c0 0x7f950c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:31.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.633+0000 7f952b400640 1 -- 172.21.15.143:0/3092072909 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f950c005180 msgr2=0x7f950c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:31.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.633+0000 7f952b400640 1 --2- 172.21.15.143:0/3092072909 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f950c005180 0x7f950c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:31.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.633+0000 7f952b400640 1 -- 172.21.15.143:0/3092072909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f950c0047c0 msgr2=0x7f950c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:31.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.633+0000 7f952b400640 1 --2- 172.21.15.143:0/3092072909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f950c0047c0 0x7f950c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:31.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.633+0000 7f952b400640 1 -- 172.21.15.143:0/3092072909 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f950c003520 con 0x7f950c003970 2024-01-16T15:02:31.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.634+0000 7f952be00640 1 -- 172.21.15.143:0/3092072909 reap_dead start 2024-01-16T15:02:31.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.635+0000 7f952b400640 1 --2- 172.21.15.143:0/3092072909 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f950c003970 0x7f950c003d70 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f951c001eb0 tx=0x7f950c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d09814499e5ae033 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:31.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.635+0000 7f952a000640 1 -- 172.21.15.143:0/3092072909 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c005ad0 con 0x7f950c003970 2024-01-16T15:02:31.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.635+0000 7f952a000640 1 -- 172.21.15.143:0/3092072909 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f951c005c50 con 0x7f950c003970 2024-01-16T15:02:31.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.635+0000 7f952a000640 1 -- 172.21.15.143:0/3092072909 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c02c890 con 0x7f950c003970 2024-01-16T15:02:31.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.635+0000 7f9532800640 1 -- 172.21.15.143:0/3092072909 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f950c003970 msgr2=0x7f950c003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:31.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.635+0000 7f9532800640 1 --2- 172.21.15.143:0/3092072909 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f950c003970 0x7f950c003d70 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f951c001eb0 tx=0x7f950c0057f0 comp rx=0 tx=0).stop 2024-01-16T15:02:31.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.636+0000 7f9532800640 1 -- 172.21.15.143:0/3092072909 shutdown_connections 2024-01-16T15:02:31.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.636+0000 7f9532800640 1 --2- 172.21.15.143:0/3092072909 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f950c003970 0x7f950c003d70 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:31.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.636+0000 7f9532800640 1 -- 172.21.15.143:0/3092072909 >> 172.21.15.143:0/3092072909 conn(0x7f952c06a920 msgr2=0x7f952c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:31.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.636+0000 7f9532800640 1 -- 172.21.15.143:0/3092072909 shutdown_connections 2024-01-16T15:02:31.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.636+0000 7f9532800640 1 -- 172.21.15.143:0/3092072909 wait complete. 2024-01-16T15:02:31.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.637+0000 7f9532800640 1 Processor -- start 2024-01-16T15:02:31.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.638+0000 7f9532800640 1 -- start start 2024-01-16T15:02:31.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.638+0000 7f9532800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f950c003970 0x7f952c153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:31.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.638+0000 7f9532800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f952c150e70 0x7f952c1512f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:31.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.638+0000 7f9532800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f952c148f40 0x7f952c1493c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:31.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.638+0000 7f9532800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952c1521d0 con 0x7f952c148f40 2024-01-16T15:02:31.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.638+0000 7f9532800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952c159900 con 0x7f952c150e70 2024-01-16T15:02:31.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.638+0000 7f9532800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f952c155de0 con 0x7f950c003970 2024-01-16T15:02:31.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.639+0000 7f952be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f952c148f40 0x7f952c1493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:31.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.639+0000 7f952be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f952c148f40 0x7f952c1493c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57966/0 (socket says 172.21.15.143:57966) 2024-01-16T15:02:31.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.639+0000 7f952be00640 1 -- 172.21.15.143:0/2360998155 learned_addr learned my addr 172.21.15.143:0/2360998155 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:02:31.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.639+0000 7f952be00640 1 -- 172.21.15.143:0/2360998155 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f950c003970 msgr2=0x7f952c153ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:31.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.639+0000 7f952b400640 1 --2- 172.21.15.143:0/2360998155 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f950c003970 0x7f952c153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:31.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.639+0000 7f952aa00640 1 --2- 172.21.15.143:0/2360998155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f952c150e70 0x7f952c1512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:31.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.639+0000 7f952be00640 1 --2- 172.21.15.143:0/2360998155 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f950c003970 0x7f952c153ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:31.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.639+0000 7f952be00640 1 -- 172.21.15.143:0/2360998155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f952c150e70 msgr2=0x7f952c1512f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:31.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.639+0000 7f952be00640 1 --2- 172.21.15.143:0/2360998155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f952c150e70 0x7f952c1512f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:31.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.639+0000 7f952be00640 1 -- 172.21.15.143:0/2360998155 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952c151930 con 0x7f952c148f40 2024-01-16T15:02:31.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.639+0000 7f952be00640 1 --2- 172.21.15.143:0/2360998155 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f952c148f40 0x7f952c1493c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f952c0b5270 tx=0x7f952400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:31.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.640+0000 7f9523400640 1 -- 172.21.15.143:0/2360998155 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95240070d0 con 0x7f952c148f40 2024-01-16T15:02:31.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.640+0000 7f9523400640 1 -- 172.21.15.143:0/2360998155 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9524007250 con 0x7f952c148f40 2024-01-16T15:02:31.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.640+0000 7f9523400640 1 -- 172.21.15.143:0/2360998155 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9524007600 con 0x7f952c148f40 2024-01-16T15:02:31.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.640+0000 7f9532800640 1 -- 172.21.15.143:0/2360998155 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f952c151c10 con 0x7f952c148f40 2024-01-16T15:02:31.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.640+0000 7f9532800640 1 -- 172.21.15.143:0/2360998155 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f952c149900 con 0x7f952c148f40 2024-01-16T15:02:31.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.641+0000 7f9532800640 1 -- 172.21.15.143:0/2360998155 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f952c067470 con 0x7f952c148f40 2024-01-16T15:02:31.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.642+0000 7f9523400640 1 -- 172.21.15.143:0/2360998155 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 89794+0+0 (secure 0 0 0) 0x7f9524007760 con 0x7f952c148f40 2024-01-16T15:02:31.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.643+0000 7f9523400640 1 --2- 172.21.15.143:0/2360998155 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f950406b590 0x7f950406da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:31.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.643+0000 7f9523400640 1 -- 172.21.15.143:0/2360998155 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9524089320 con 0x7f952c148f40 2024-01-16T15:02:31.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.643+0000 7f952b400640 1 --2- 172.21.15.143:0/2360998155 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f950406b590 0x7f950406da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:31.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.644+0000 7f952b400640 1 --2- 172.21.15.143:0/2360998155 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f950406b590 0x7f950406da50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f951c034000 tx=0x7f951c009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:31.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.648+0000 7f9523400640 1 -- 172.21.15.143:0/2360998155 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9524054060 con 0x7f952c148f40 2024-01-16T15:02:31.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.840+0000 7f9532800640 1 -- 172.21.15.143:0/2360998155 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f952c1596d0 con 0x7f952c148f40 2024-01-16T15:02:31.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.841+0000 7f9523400640 1 -- 172.21.15.143:0/2360998155 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+13048 (secure 0 0 0) 0x7f9524057680 con 0x7f952c148f40 2024-01-16T15:02:31.820 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-16T15:02:31.821 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":39,"fsid":"c91ab77b-ad1a-4af4-9a1d-13881fe5c1d0","created":"2024-01-16T14:59:38.532601+0000","modified":"2024-01-16T15:02:31.038150+0000","last_up_change":"2024-01-16T14:59:50.983699+0000","last_in_change":"2024-01-16T14:59:42.056297+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:59:51.727720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f4c8891d-ab29-472a-907d-8446f20f7012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6801","nonce":1569293626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6805","nonce":1569293626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6814","nonce":1569293626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":1569293626},{"type":"v1","addr":"172.21.15.143:6809","nonce":1569293626}]},"public_addr":"172.21.15.143:6801/1569293626","cluster_addr":"172.21.15.143:6805/1569293626","heartbeat_back_addr":"172.21.15.143:6814/1569293626","heartbeat_front_addr":"172.21.15.143:6809/1569293626","state":["exists","up"]},{"osd":1,"uuid":"c4a2788d-8b89-4992-82b2-7f4b9debd509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6804","nonce":1276841168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6808","nonce":1276841168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6813","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6815","nonce":1276841168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1276841168},{"type":"v1","addr":"172.21.15.143:6811","nonce":1276841168}]},"public_addr":"172.21.15.143:6804/1276841168","cluster_addr":"172.21.15.143:6808/1276841168","heartbeat_back_addr":"172.21.15.143:6815/1276841168","heartbeat_front_addr":"172.21.15.143:6811/1276841168","state":["exists","up"]},{"osd":2,"uuid":"64645c1d-31b2-4794-b0f2-7ccf8df30393","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6817","nonce":1217670170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6819","nonce":1217670170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6823","nonce":1217670170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1217670170},{"type":"v1","addr":"172.21.15.143:6821","nonce":1217670170}]},"public_addr":"172.21.15.143:6817/1217670170","cluster_addr":"172.21.15.143:6819/1217670170","heartbeat_back_addr":"172.21.15.143:6823/1217670170","heartbeat_front_addr":"172.21.15.143:6821/1217670170","state":["exists","up"]},{"osd":3,"uuid":"73674129-a84e-4d31-8247-7d0179ae6df2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6825","nonce":3414188012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6827","nonce":3414188012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6831","nonce":3414188012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3414188012},{"type":"v1","addr":"172.21.15.143:6829","nonce":3414188012}]},"public_addr":"172.21.15.143:6825/3414188012","cluster_addr":"172.21.15.143:6827/3414188012","heartbeat_back_addr":"172.21.15.143:6831/3414188012","heartbeat_front_addr":"172.21.15.143:6829/3414188012","state":["exists","up"]},{"osd":4,"uuid":"cea28c9b-2f1c-4087-b415-9960789131d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6811","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6812","nonce":1070255716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6816","nonce":1070255716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6823","nonce":1070255716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1070255716},{"type":"v1","addr":"172.21.15.204:6820","nonce":1070255716}]},"public_addr":"172.21.15.204:6812/1070255716","cluster_addr":"172.21.15.204:6816/1070255716","heartbeat_back_addr":"172.21.15.204:6823/1070255716","heartbeat_front_addr":"172.21.15.204:6820/1070255716","state":["exists","up"]},{"osd":5,"uuid":"33b63560-5574-4e78-983f-b398d67a2cb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6809","nonce":3605417837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6813","nonce":3605417837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6819","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6821","nonce":3605417837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":3605417837},{"type":"v1","addr":"172.21.15.204:6817","nonce":3605417837}]},"public_addr":"172.21.15.204:6809/3605417837","cluster_addr":"172.21.15.204:6813/3605417837","heartbeat_back_addr":"172.21.15.204:6821/3605417837","heartbeat_front_addr":"172.21.15.204:6817/3605417837","state":["exists","up"]},{"osd":6,"uuid":"df397427-8940-4eee-9c79-eece669d735d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6801","nonce":1085000534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6803","nonce":1085000534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6808","nonce":1085000534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1085000534},{"type":"v1","addr":"172.21.15.204:6805","nonce":1085000534}]},"public_addr":"172.21.15.204:6801/1085000534","cluster_addr":"172.21.15.204:6803/1085000534","heartbeat_back_addr":"172.21.15.204:6808/1085000534","heartbeat_front_addr":"172.21.15.204:6805/1085000534","state":["exists","up"]},{"osd":7,"uuid":"a2e5a15c-8312-405a-ac02-f7e1fc9df2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6825","nonce":1337220609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6827","nonce":1337220609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6831","nonce":1337220609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1337220609},{"type":"v1","addr":"172.21.15.204:6829","nonce":1337220609}]},"public_addr":"172.21.15.204:6825/1337220609","cluster_addr":"172.21.15.204:6827/1337220609","heartbeat_back_addr":"172.21.15.204:6831/1337220609","heartbeat_front_addr":"172.21.15.204:6829/1337220609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:49.984507+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.395306+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.632998+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.373627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.342920+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.438366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.331926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:59:47.406657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:6837/4104468169":"2024-01-17T15:02:28.025330+0000","172.21.15.143:6836/4104468169":"2024-01-17T15:02:28.025330+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T15:02:31.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.843+0000 7f9532800640 1 -- 172.21.15.143:0/2360998155 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f950406b590 msgr2=0x7f950406da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:31.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.843+0000 7f9532800640 1 --2- 172.21.15.143:0/2360998155 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f950406b590 0x7f950406da50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f951c034000 tx=0x7f951c009030 comp rx=0 tx=0).stop 2024-01-16T15:02:31.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.843+0000 7f9532800640 1 -- 172.21.15.143:0/2360998155 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f952c148f40 msgr2=0x7f952c1493c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:31.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.843+0000 7f9532800640 1 --2- 172.21.15.143:0/2360998155 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f952c148f40 0x7f952c1493c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f952c0b5270 tx=0x7f952400c960 comp rx=0 tx=0).stop 2024-01-16T15:02:31.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.844+0000 7f9532800640 1 -- 172.21.15.143:0/2360998155 shutdown_connections 2024-01-16T15:02:31.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.844+0000 7f9532800640 1 --2- 172.21.15.143:0/2360998155 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f950406b590 0x7f950406da50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:31.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.844+0000 7f9532800640 1 --2- 172.21.15.143:0/2360998155 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f952c148f40 0x7f952c1493c0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:31.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.844+0000 7f9532800640 1 --2- 172.21.15.143:0/2360998155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f952c150e70 0x7f952c1512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:31.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.844+0000 7f9532800640 1 --2- 172.21.15.143:0/2360998155 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f950c003970 0x7f952c153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:31.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.844+0000 7f9532800640 1 -- 172.21.15.143:0/2360998155 >> 172.21.15.143:0/2360998155 conn(0x7f952c06a920 msgr2=0x7f952c0af580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:31.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.844+0000 7f9532800640 1 -- 172.21.15.143:0/2360998155 shutdown_connections 2024-01-16T15:02:31.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:31.844+0000 7f9532800640 1 -- 172.21.15.143:0/2360998155 wait complete. 2024-01-16T15:02:31.841 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-16T14:59:51.727720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-16T15:02:31.842 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-16T15:02:32.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.075+0000 7f157ac00640 1 Processor -- start 2024-01-16T15:02:32.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.075+0000 7f157ac00640 1 -- start start 2024-01-16T15:02:32.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.077+0000 7f157ac00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f157415d270 0x7f157415d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:32.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.077+0000 7f157ac00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1574161260 0x7f15741616c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:32.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.077+0000 7f157ac00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f157415db90 con 0x7f15740ceb10 2024-01-16T15:02:32.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.077+0000 7f157ac00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1574161ce0 con 0x7f1574161260 2024-01-16T15:02:32.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.077+0000 7f157ac00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1574161fc0 con 0x7f157415d270 2024-01-16T15:02:32.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.077+0000 7f1573400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1574161260 0x7f15741616c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:32.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.077+0000 7f1578c00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f15740ceb10 0x7f15740d19b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33968/0 (socket says 172.21.15.143:33968) 2024-01-16T15:02:32.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.077+0000 7f1578c00640 1 -- 172.21.15.143:0/2614161908 learned_addr learned my addr 172.21.15.143:0/2614161908 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:02:32.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.077+0000 7f1573400640 1 -- 172.21.15.143:0/2614161908 >> v2:172.21.15.204:3300/0 conn(0x7f157415d270 msgr2=0x7f157415d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:32.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.077+0000 7f1573400640 1 --2- 172.21.15.143:0/2614161908 >> v2:172.21.15.204:3300/0 conn(0x7f157415d270 0x7f157415d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:32.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.077+0000 7f1573400640 1 -- 172.21.15.143:0/2614161908 >> v1:172.21.15.143:6789/0 conn(0x7f15740ceb10 legacy=0x7f15740d19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:32.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.077+0000 7f1573400640 1 -- 172.21.15.143:0/2614161908 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15740ce310 con 0x7f1574161260 2024-01-16T15:02:32.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.077+0000 7f1572a00640 1 -- 172.21.15.143:0/2614161908 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2544998565 0 0) 0x7f157415db90 con 0x7f15740ceb10 2024-01-16T15:02:32.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.079+0000 7f1573400640 1 --2- 172.21.15.143:0/2614161908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1574161260 0x7f15741616c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f15740b5f50 tx=0x7f155c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e53286a61b859b38 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:32.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.079+0000 7f1572a00640 1 -- 172.21.15.143:0/2614161908 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c031830 con 0x7f1574161260 2024-01-16T15:02:32.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.079+0000 7f1572a00640 1 -- 172.21.15.143:0/2614161908 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f155c031990 con 0x7f1574161260 2024-01-16T15:02:32.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.079+0000 7f1572a00640 1 -- 172.21.15.143:0/2614161908 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c031d20 con 0x7f1574161260 2024-01-16T15:02:32.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.079+0000 7f157ac00640 1 -- 172.21.15.143:0/2614161908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1574161260 msgr2=0x7f15741616c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:32.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.079+0000 7f157ac00640 1 --2- 172.21.15.143:0/2614161908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1574161260 0x7f15741616c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f15740b5f50 tx=0x7f155c02b8d0 comp rx=0 tx=0).stop 2024-01-16T15:02:32.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.079+0000 7f157ac00640 1 -- 172.21.15.143:0/2614161908 shutdown_connections 2024-01-16T15:02:32.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.079+0000 7f157ac00640 1 --2- 172.21.15.143:0/2614161908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1574161260 0x7f15741616c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:32.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.079+0000 7f157ac00640 1 --2- 172.21.15.143:0/2614161908 >> v2:172.21.15.204:3300/0 conn(0x7f157415d270 0x7f157415d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:32.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.079+0000 7f157ac00640 1 -- 172.21.15.143:0/2614161908 >> 172.21.15.143:0/2614161908 conn(0x7f157408af60 msgr2=0x7f157409c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:32.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.080+0000 7f157ac00640 1 -- 172.21.15.143:0/2614161908 shutdown_connections 2024-01-16T15:02:32.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.080+0000 7f157ac00640 1 -- 172.21.15.143:0/2614161908 wait complete. 2024-01-16T15:02:32.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.081+0000 7f157ac00640 1 Processor -- start 2024-01-16T15:02:32.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.081+0000 7f157ac00640 1 -- start start 2024-01-16T15:02:32.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.082+0000 7f157ac00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15740ceb10 0x7f15741f70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:32.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.082+0000 7f157ac00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f157415d270 0x7f1574159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:32.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.082+0000 7f157ac00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1574161260 0x7f15740b3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:32.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.082+0000 7f157ac00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15740c75a0 con 0x7f1574161260 2024-01-16T15:02:32.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.082+0000 7f157ac00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15741f7600 con 0x7f15740ceb10 2024-01-16T15:02:32.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.082+0000 7f157ac00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15740c8500 con 0x7f157415d270 2024-01-16T15:02:32.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.083+0000 7f1578c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1574161260 0x7f15740b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:32.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.083+0000 7f1578c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1574161260 0x7f15740b3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57978/0 (socket says 172.21.15.143:57978) 2024-01-16T15:02:32.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.083+0000 7f1578c00640 1 -- 172.21.15.143:0/495404886 learned_addr learned my addr 172.21.15.143:0/495404886 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:02:32.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.083+0000 7f1573e00640 1 --2- 172.21.15.143:0/495404886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15740ceb10 0x7f15741f70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:32.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.083+0000 7f1578c00640 1 -- 172.21.15.143:0/495404886 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f157415d270 msgr2=0x7f1574159ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:32.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.083+0000 7f1578c00640 1 --2- 172.21.15.143:0/495404886 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f157415d270 0x7f1574159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:32.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.083+0000 7f1578c00640 1 -- 172.21.15.143:0/495404886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15740ceb10 msgr2=0x7f15741f70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:32.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.083+0000 7f1578c00640 1 --2- 172.21.15.143:0/495404886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15740ceb10 0x7f15741f70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:32.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.083+0000 7f1578c00640 1 -- 172.21.15.143:0/495404886 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f155c009400 con 0x7f1574161260 2024-01-16T15:02:32.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.083+0000 7f1573e00640 1 --2- 172.21.15.143:0/495404886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15740ceb10 0x7f15741f70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T15:02:32.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.084+0000 7f1578c00640 1 --2- 172.21.15.143:0/495404886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1574161260 0x7f15740b3740 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f156c00c780 tx=0x7f156c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:32.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.084+0000 7f156be00640 1 -- 172.21.15.143:0/495404886 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c00ce90 con 0x7f1574161260 2024-01-16T15:02:32.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.084+0000 7f156be00640 1 -- 172.21.15.143:0/495404886 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156c00a640 con 0x7f1574161260 2024-01-16T15:02:32.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.084+0000 7f156be00640 1 -- 172.21.15.143:0/495404886 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c00a9d0 con 0x7f1574161260 2024-01-16T15:02:32.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.084+0000 7f157ac00640 1 -- 172.21.15.143:0/495404886 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15740c87e0 con 0x7f1574161260 2024-01-16T15:02:32.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.084+0000 7f157ac00640 1 -- 172.21.15.143:0/495404886 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15740c8da0 con 0x7f1574161260 2024-01-16T15:02:32.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.085+0000 7f157ac00640 1 -- 172.21.15.143:0/495404886 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15740d11e0 con 0x7f1574161260 2024-01-16T15:02:32.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.086+0000 7f156be00640 1 -- 172.21.15.143:0/495404886 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 89794+0+0 (secure 0 0 0) 0x7f156c00ab30 con 0x7f1574161260 2024-01-16T15:02:32.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.087+0000 7f156be00640 1 --2- 172.21.15.143:0/495404886 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f154c06b590 0x7f154c06da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:32.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.087+0000 7f156be00640 1 -- 172.21.15.143:0/495404886 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f156c0892d0 con 0x7f1574161260 2024-01-16T15:02:32.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.087+0000 7f1573e00640 1 --2- 172.21.15.143:0/495404886 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f154c06b590 0x7f154c06da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:32.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.088+0000 7f1573e00640 1 --2- 172.21.15.143:0/495404886 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f154c06b590 0x7f154c06da50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1564006fd0 tx=0x7f1564008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:32.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.091+0000 7f156be00640 1 -- 172.21.15.143:0/495404886 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f156c054060 con 0x7f1574161260 2024-01-16T15:02:32.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.283+0000 7f157ac00640 1 -- 172.21.15.143:0/495404886 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f157406b7a0 con 0x7f1574161260 2024-01-16T15:02:32.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.284+0000 7f156be00640 1 -- 172.21.15.143:0/495404886 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) v1 ==== 93+0+10 (secure 0 0 0) 0x7f156c018020 con 0x7f1574161260 2024-01-16T15:02:32.263 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-16T15:02:32.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.286+0000 7f157ac00640 1 -- 172.21.15.143:0/495404886 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f154c06b590 msgr2=0x7f154c06da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:32.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.286+0000 7f157ac00640 1 --2- 172.21.15.143:0/495404886 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f154c06b590 0x7f154c06da50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1564006fd0 tx=0x7f1564008040 comp rx=0 tx=0).stop 2024-01-16T15:02:32.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.286+0000 7f157ac00640 1 -- 172.21.15.143:0/495404886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1574161260 msgr2=0x7f15740b3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:32.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.286+0000 7f157ac00640 1 --2- 172.21.15.143:0/495404886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1574161260 0x7f15740b3740 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f156c00c780 tx=0x7f156c00cc40 comp rx=0 tx=0).stop 2024-01-16T15:02:32.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.287+0000 7f157ac00640 1 -- 172.21.15.143:0/495404886 shutdown_connections 2024-01-16T15:02:32.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.287+0000 7f157ac00640 1 --2- 172.21.15.143:0/495404886 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7f154c06b590 0x7f154c06da50 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:32.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.287+0000 7f157ac00640 1 --2- 172.21.15.143:0/495404886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1574161260 0x7f15740b3740 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:32.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.287+0000 7f157ac00640 1 --2- 172.21.15.143:0/495404886 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f157415d270 0x7f1574159ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:32.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.287+0000 7f157ac00640 1 --2- 172.21.15.143:0/495404886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15740ceb10 0x7f15741f70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:32.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.287+0000 7f157ac00640 1 -- 172.21.15.143:0/495404886 >> 172.21.15.143:0/495404886 conn(0x7f157408af60 msgr2=0x7f15740b45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:32.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.287+0000 7f157ac00640 1 -- 172.21.15.143:0/495404886 shutdown_connections 2024-01-16T15:02:32.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.287+0000 7f157ac00640 1 -- 172.21.15.143:0/495404886 wait complete. 2024-01-16T15:02:32.286 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-16T15:02:32.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.516+0000 7fa5d8c00640 1 Processor -- start 2024-01-16T15:02:32.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.516+0000 7fa5d8c00640 1 -- start start 2024-01-16T15:02:32.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.518+0000 7fa5d8c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa5d415a1c0 0x7fa5d4070ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:32.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.518+0000 7fa5d8c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa5d40b15e0 0x7fa5d415dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:32.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.518+0000 7fa5d8c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d415e2e0 con 0x7fa5d406d640 2024-01-16T15:02:32.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.518+0000 7fa5d8c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5d40cd920 con 0x7fa5d415a1c0 2024-01-16T15:02:32.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.518+0000 7fa5d8c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5d40b1a50 con 0x7fa5d40b15e0 2024-01-16T15:02:32.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.518+0000 7fa5d3400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa5d415a1c0 0x7fa5d4070ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:32.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.518+0000 7fa5d3e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fa5d406d640 0x7fa5d406da50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33970/0 (socket says 172.21.15.143:33970) 2024-01-16T15:02:32.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.518+0000 7fa5d3e00640 1 -- 172.21.15.143:0/651732073 learned_addr learned my addr 172.21.15.143:0/651732073 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-16T15:02:32.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.518+0000 7fa5cbe00640 1 --2- 172.21.15.143:0/651732073 >> v2:172.21.15.204:3300/0 conn(0x7fa5d40b15e0 0x7fa5d415dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:32.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.519+0000 7fa5d2a00640 1 -- 172.21.15.143:0/651732073 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4224416701 0 0) 0x7fa5d415e2e0 con 0x7fa5d406d640 2024-01-16T15:02:32.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.519+0000 7fa5d2a00640 1 -- 172.21.15.143:0/651732073 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5b4003520 con 0x7fa5d406d640 2024-01-16T15:02:32.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.519+0000 7fa5d3400640 1 -- 172.21.15.143:0/651732073 >> v2:172.21.15.204:3300/0 conn(0x7fa5d40b15e0 msgr2=0x7fa5d415dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T15:02:32.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.519+0000 7fa5d3400640 1 --2- 172.21.15.143:0/651732073 >> v2:172.21.15.204:3300/0 conn(0x7fa5d40b15e0 0x7fa5d415dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:32.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.519+0000 7fa5d3400640 1 -- 172.21.15.143:0/651732073 >> v1:172.21.15.143:6789/0 conn(0x7fa5d406d640 legacy=0x7fa5d406da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:32.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.519+0000 7fa5d3400640 1 -- 172.21.15.143:0/651732073 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d415e580 con 0x7fa5d415a1c0 2024-01-16T15:02:32.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.520+0000 7fa5d3400640 1 --2- 172.21.15.143:0/651732073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5d415a1c0 0x7fa5d4070ea0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa5d40b5180 tx=0x7fa5c402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e984c6c47240b16 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:32.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.520+0000 7fa5d2a00640 1 -- 172.21.15.143:0/651732073 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4030610 con 0x7fa5d415a1c0 2024-01-16T15:02:32.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.520+0000 7fa5d2a00640 1 -- 172.21.15.143:0/651732073 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c4030770 con 0x7fa5d415a1c0 2024-01-16T15:02:32.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.520+0000 7fa5d2a00640 1 -- 172.21.15.143:0/651732073 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4030b00 con 0x7fa5d415a1c0 2024-01-16T15:02:32.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.521+0000 7fa5d8c00640 1 -- 172.21.15.143:0/651732073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5d415a1c0 msgr2=0x7fa5d4070ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:32.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.521+0000 7fa5d8c00640 1 --2- 172.21.15.143:0/651732073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5d415a1c0 0x7fa5d4070ea0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa5d40b5180 tx=0x7fa5c402b960 comp rx=0 tx=0).stop 2024-01-16T15:02:32.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.521+0000 7fa5d8c00640 1 -- 172.21.15.143:0/651732073 shutdown_connections 2024-01-16T15:02:32.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.521+0000 7fa5d8c00640 1 --2- 172.21.15.143:0/651732073 >> v2:172.21.15.204:3300/0 conn(0x7fa5d40b15e0 0x7fa5d415dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:32.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.521+0000 7fa5d8c00640 1 --2- 172.21.15.143:0/651732073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5d415a1c0 0x7fa5d4070ea0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:32.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.521+0000 7fa5d8c00640 1 -- 172.21.15.143:0/651732073 >> 172.21.15.143:0/651732073 conn(0x7fa5d40cbb40 msgr2=0x7fa5d40cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:32.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.521+0000 7fa5d8c00640 1 -- 172.21.15.143:0/651732073 shutdown_connections 2024-01-16T15:02:32.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.521+0000 7fa5d8c00640 1 -- 172.21.15.143:0/651732073 wait complete. 2024-01-16T15:02:32.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.523+0000 7fa5d8c00640 1 Processor -- start 2024-01-16T15:02:32.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.523+0000 7fa5d8c00640 1 -- start start 2024-01-16T15:02:32.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.524+0000 7fa5d8c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa5d406d640 0x7fa5d4153570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:32.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.524+0000 7fa5d8c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5d40b15e0 0x7fa5d415be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:32.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.524+0000 7fa5d8c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5d415a1c0 0x7fa5d415c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:32.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.524+0000 7fa5d8c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d414f960 con 0x7fa5d40b15e0 2024-01-16T15:02:32.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.524+0000 7fa5d8c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d414fc40 con 0x7fa5d415a1c0 2024-01-16T15:02:32.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.524+0000 7fa5d8c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d414ff20 con 0x7fa5d406d640 2024-01-16T15:02:32.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.524+0000 7fa5cbe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5d40b15e0 0x7fa5d415be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:32.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.524+0000 7fa5cbe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5d40b15e0 0x7fa5d415be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57988/0 (socket says 172.21.15.143:57988) 2024-01-16T15:02:32.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.524+0000 7fa5cbe00640 1 -- 172.21.15.143:0/533427629 learned_addr learned my addr 172.21.15.143:0/533427629 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-16T15:02:32.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.525+0000 7fa5d3400640 1 --2- 172.21.15.143:0/533427629 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa5d406d640 0x7fa5d4153570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:32.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.525+0000 7fa5cbe00640 1 -- 172.21.15.143:0/533427629 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa5d406d640 msgr2=0x7fa5d4153570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:32.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.525+0000 7fa5cbe00640 1 --2- 172.21.15.143:0/533427629 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa5d406d640 0x7fa5d4153570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:32.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.525+0000 7fa5d3e00640 1 --2- 172.21.15.143:0/533427629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5d415a1c0 0x7fa5d415c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:32.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.525+0000 7fa5cbe00640 1 -- 172.21.15.143:0/533427629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5d415a1c0 msgr2=0x7fa5d415c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:32.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.525+0000 7fa5cbe00640 1 --2- 172.21.15.143:0/533427629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5d415a1c0 0x7fa5d415c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:32.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.525+0000 7fa5cbe00640 1 -- 172.21.15.143:0/533427629 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c4009400 con 0x7fa5d40b15e0 2024-01-16T15:02:32.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.525+0000 7fa5cbe00640 1 --2- 172.21.15.143:0/533427629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5d40b15e0 0x7fa5d415be50 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00d800 tx=0x7fa5bc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:32.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.525+0000 7fa5cb400640 1 -- 172.21.15.143:0/533427629 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc00b6c0 con 0x7fa5d40b15e0 2024-01-16T15:02:32.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.525+0000 7fa5cb400640 1 -- 172.21.15.143:0/533427629 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5bc00b820 con 0x7fa5d40b15e0 2024-01-16T15:02:32.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.525+0000 7fa5cb400640 1 -- 172.21.15.143:0/533427629 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc00bbb0 con 0x7fa5d40b15e0 2024-01-16T15:02:32.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.525+0000 7fa5d8c00640 1 -- 172.21.15.143:0/533427629 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5d4150200 con 0x7fa5d40b15e0 2024-01-16T15:02:32.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.525+0000 7fa5d8c00640 1 -- 172.21.15.143:0/533427629 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5d414f580 con 0x7fa5d40b15e0 2024-01-16T15:02:32.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.527+0000 7fa5d8c00640 1 -- 172.21.15.143:0/533427629 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5d4073e80 con 0x7fa5d40b15e0 2024-01-16T15:02:32.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.528+0000 7fa5cb400640 1 -- 172.21.15.143:0/533427629 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 89794+0+0 (secure 0 0 0) 0x7fa5bc00bd10 con 0x7fa5d40b15e0 2024-01-16T15:02:32.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.528+0000 7fa5cb400640 1 --2- 172.21.15.143:0/533427629 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fa5ac06b590 0x7fa5ac06da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T15:02:32.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.529+0000 7fa5cb400640 1 -- 172.21.15.143:0/533427629 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa5bc08a2c0 con 0x7fa5d40b15e0 2024-01-16T15:02:32.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.529+0000 7fa5d3400640 1 --2- 172.21.15.143:0/533427629 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fa5ac06b590 0x7fa5ac06da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T15:02:32.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.530+0000 7fa5d3400640 1 --2- 172.21.15.143:0/533427629 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fa5ac06b590 0x7fa5ac06da50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4036000 tx=0x7fa5c403b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T15:02:32.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.533+0000 7fa5cb400640 1 -- 172.21.15.143:0/533427629 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa5bc0550e0 con 0x7fa5d40b15e0 2024-01-16T15:02:32.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.735+0000 7fa5d8c00640 1 -- 172.21.15.143:0/533427629 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa5d404e840 con 0x7fa5d40b15e0 2024-01-16T15:02:32.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.736+0000 7fa5cb400640 1 -- 172.21.15.143:0/533427629 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v39) v1 ==== 80+0+124 (secure 0 0 0) 0x7fa5bc058700 con 0x7fa5d40b15e0 2024-01-16T15:02:32.715 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6837/4104468169 2024-01-17T15:02:28.025330+0000 2024-01-16T15:02:32.715 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6836/4104468169 2024-01-17T15:02:28.025330+0000 2024-01-16T15:02:32.716 INFO:teuthology.orchestra.run.smithi143.stderr:listed 2 entries 2024-01-16T15:02:32.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.738+0000 7fa5d8c00640 1 -- 172.21.15.143:0/533427629 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fa5ac06b590 msgr2=0x7fa5ac06da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:32.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.739+0000 7fa5d8c00640 1 --2- 172.21.15.143:0/533427629 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fa5ac06b590 0x7fa5ac06da50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4036000 tx=0x7fa5c403b040 comp rx=0 tx=0).stop 2024-01-16T15:02:32.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.739+0000 7fa5d8c00640 1 -- 172.21.15.143:0/533427629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5d40b15e0 msgr2=0x7fa5d415be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T15:02:32.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.739+0000 7fa5d8c00640 1 --2- 172.21.15.143:0/533427629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5d40b15e0 0x7fa5d415be50 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00d800 tx=0x7fa5bc00dcc0 comp rx=0 tx=0).stop 2024-01-16T15:02:32.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.739+0000 7fa5d8c00640 1 -- 172.21.15.143:0/533427629 shutdown_connections 2024-01-16T15:02:32.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.739+0000 7fa5d8c00640 1 --2- 172.21.15.143:0/533427629 >> [v2:172.21.15.143:6832/3153492115,v1:172.21.15.143:6833/3153492115] conn(0x7fa5ac06b590 0x7fa5ac06da50 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:32.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.739+0000 7fa5d8c00640 1 --2- 172.21.15.143:0/533427629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5d415a1c0 0x7fa5d415c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:32.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.739+0000 7fa5d8c00640 1 --2- 172.21.15.143:0/533427629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5d40b15e0 0x7fa5d415be50 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:32.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.739+0000 7fa5d8c00640 1 --2- 172.21.15.143:0/533427629 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa5d406d640 0x7fa5d4153570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T15:02:32.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.739+0000 7fa5d8c00640 1 -- 172.21.15.143:0/533427629 >> 172.21.15.143:0/533427629 conn(0x7fa5d40cbb40 msgr2=0x7fa5d40acc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T15:02:32.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.739+0000 7fa5d8c00640 1 -- 172.21.15.143:0/533427629 shutdown_connections 2024-01-16T15:02:32.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-16T15:02:32.739+0000 7fa5d8c00640 1 -- 172.21.15.143:0/533427629 wait complete. 2024-01-16T15:02:32.737 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-16T15:02:32.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T15:02:32.738 DEBUG:teuthology.orchestra.run.smithi143:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-01-16T15:02:53.722 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:02:53.732 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:03:23.862 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:03:23.873 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:03:54.005 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:03:54.014 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:04:24.148 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:04:24.157 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:04:54.296 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:04:54.305 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:05:24.434 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:05:24.444 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:05:54.591 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:05:54.600 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:06:24.745 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:06:24.754 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:06:54.893 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:06:54.901 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:07:25.032 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:07:25.041 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:07:32.746 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 745, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/virtualenv/lib/python3.8/site-packages/paramiko/buffered_pipe.py", line 154, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 908, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 109, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 275, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 1374, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 747, in recv_stderr raise socket.timeout() socket.timeout 2024-01-16T15:07:32.748 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 697, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/virtualenv/lib/python3.8/site-packages/paramiko/buffered_pipe.py", line 154, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 908, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 109, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 275, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 1359, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 699, in recv raise socket.timeout() socket.timeout 2024-01-16T15:07:55.171 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:07:55.179 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:08:25.310 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:08:25.319 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:08:55.467 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:08:55.475 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:09:25.620 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:09:25.628 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:09:55.756 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:09:55.764 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:10:25.901 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:10:25.910 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:10:56.042 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:10:56.051 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:11:26.183 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:11:26.191 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:11:56.340 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:11:56.349 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:12:26.489 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:12:26.497 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:12:56.627 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:12:56.636 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:13:26.766 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:13:26.783 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:13:56.915 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:13:56.924 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:14:27.051 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:14:27.060 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:14:57.193 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:14:57.202 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:15:27.343 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:15:27.352 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:15:57.484 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:15:57.493 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:16:27.633 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:16:27.642 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:16:57.782 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:16:57.791 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:17:27.923 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:17:27.932 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:17:58.064 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:17:58.072 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:18:28.205 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:18:28.214 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:18:58.345 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:18:58.354 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:19:28.480 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:19:28.489 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:19:58.627 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:19:58.636 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:20:28.775 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:20:28.784 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:20:58.916 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:20:58.925 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:21:29.073 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:21:29.082 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:21:59.214 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:21:59.223 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:22:29.363 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:22:29.372 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:22:59.505 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:22:59.514 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:23:29.653 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:23:29.664 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:23:59.803 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:23:59.811 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:24:29.952 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:24:29.960 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:25:00.100 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:25:00.109 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:25:30.250 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:25:30.258 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:26:00.395 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:26:00.404 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:26:30.530 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:26:30.539 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:27:00.679 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:27:00.688 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:27:30.837 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:27:30.845 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:28:00.983 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:28:00.992 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:28:31.125 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:28:31.134 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:29:01.264 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:29:01.273 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:29:31.405 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:29:31.414 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:30:01.552 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:30:01.561 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:30:31.701 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:30:31.709 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:31:01.848 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:31:01.856 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:31:31.996 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:31:32.005 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:32:02.144 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:32:02.153 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:32:32.292 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:32:32.300 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:33:02.431 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:33:02.439 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:33:32.579 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:33:32.587 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:34:02.738 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:34:02.747 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:34:32.883 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:34:32.891 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:35:03.032 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:35:03.040 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:35:33.170 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:35:33.179 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:36:03.317 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:36:03.326 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:36:33.466 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:36:33.474 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:37:03.606 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:37:03.614 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:37:33.755 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:37:33.764 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:38:03.885 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:38:03.893 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:38:34.034 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:38:34.043 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:39:04.173 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:39:04.181 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:39:34.321 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:39:34.330 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:40:04.462 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:40:04.471 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:40:34.601 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:40:34.609 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:41:04.740 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:41:04.749 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:41:34.888 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:41:34.897 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:42:05.027 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:42:05.036 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:42:35.176 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:42:35.184 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:43:05.323 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:43:05.332 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:43:35.464 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:43:35.473 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:44:05.611 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:44:05.622 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:44:35.770 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:44:35.778 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:45:05.909 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:45:05.918 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:45:36.066 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:45:36.074 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:46:06.223 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:46:06.231 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:46:36.424 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:46:36.433 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:47:06.586 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:47:06.595 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:47:36.744 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:47:36.752 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:48:06.906 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:48:06.914 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:48:37.043 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:48:37.052 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:49:07.257 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:49:07.266 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:49:37.403 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:49:37.412 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:50:07.542 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:50:07.551 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:50:37.689 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:50:37.698 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:51:07.838 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:51:07.847 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:51:37.986 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:51:37.994 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:52:08.142 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:52:08.151 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:52:38.283 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:52:38.291 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:53:08.448 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:53:08.457 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:53:38.620 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:53:38.628 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:54:08.777 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:54:08.786 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:54:38.935 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:54:38.944 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:55:09.083 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:55:09.093 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:55:39.230 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:55:39.239 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:56:09.378 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:56:09.386 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:56:39.541 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:56:39.550 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:57:09.691 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:57:09.700 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:57:39.846 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:57:39.855 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:58:09.984 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:58:09.993 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:58:40.142 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:58:40.151 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:59:10.290 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:59:10.298 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:59:40.428 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:59:40.436 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:00:10.567 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:00:10.576 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:00:40.714 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:00:40.722 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:01:10.861 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:01:10.871 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:01:41.084 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:01:41.093 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:02:11.290 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:02:11.298 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:02:41.507 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:02:41.516 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:03:11.694 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:03:11.703 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:03:42.018 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:03:42.077 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:04:12.272 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:04:12.281 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:04:42.494 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:04:42.503 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:05:12.716 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:05:12.724 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:05:42.942 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:05:42.950 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:06:13.166 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:06:13.175 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:06:43.388 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:06:43.396 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:07:13.611 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:07:13.619 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:07:43.767 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:07:43.776 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:08:14.045 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:08:14.053 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:08:44.255 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:08:44.264 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:09:14.511 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:09:14.519 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:09:44.726 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:09:44.735 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:10:14.948 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:10:14.957 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:10:45.155 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:10:45.164 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:11:15.294 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:11:15.303 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:11:45.516 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:11:45.526 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:12:15.730 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:12:15.739 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:12:45.936 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:12:45.945 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:13:16.142 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:13:16.151 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:13:46.283 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:13:46.315 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:14:16.540 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:14:16.549 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:14:46.738 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:14:46.747 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:15:16.923 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:15:16.934 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:15:47.143 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:15:47.152 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:16:17.375 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:16:17.383 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:16:47.607 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:16:47.616 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:17:17.898 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:17:17.908 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:17:48.112 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:17:48.121 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:18:18.311 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:18:18.319 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:18:48.509 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:18:48.518 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:19:18.717 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:19:18.726 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:19:48.849 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:19:48.858 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:20:19.116 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:20:19.125 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:20:49.313 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:20:49.322 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:21:19.526 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:21:19.534 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:21:49.733 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:21:49.745 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:22:19.876 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:22:19.884 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:22:50.108 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:22:50.116 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:23:20.333 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:23:20.341 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:23:50.572 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:23:50.580 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:24:20.794 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:24:20.803 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:24:51.013 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:24:51.022 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:25:21.228 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:25:21.236 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:25:51.461 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:25:51.469 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:26:21.676 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:26:21.684 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:26:51.900 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:26:51.908 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:27:22.039 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:27:22.048 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:27:52.234 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:27:52.242 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:28:22.444 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:28:22.454 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:28:52.677 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:28:52.686 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:29:22.848 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:29:22.857 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:29:53.070 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:29:53.078 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:30:23.287 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:30:23.295 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:30:53.526 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:30:53.535 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:31:23.743 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:31:23.752 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:31:53.899 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:31:53.907 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:32:24.120 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:32:24.128 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:32:54.340 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:32:54.348 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:33:24.546 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:33:24.554 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:33:54.761 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:33:54.771 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:34:24.905 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:34:24.914 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:34:55.175 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:34:55.184 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:35:25.390 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:35:25.398 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:35:55.606 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:35:55.614 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:36:25.830 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:36:25.843 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:36:55.972 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:36:55.980 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:37:26.111 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:37:26.119 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:37:56.331 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:37:56.340 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:38:26.550 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:38:26.558 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:38:56.774 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:38:56.783 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:39:26.914 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:39:26.923 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:39:57.080 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:39:57.088 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:40:27.219 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:40:27.227 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:40:57.442 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:40:57.451 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:41:27.657 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:41:27.665 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:41:57.881 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:41:57.889 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:42:28.088 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:42:28.096 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:42:58.226 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:42:58.235 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:43:28.441 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:43:28.450 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:43:58.656 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:43:58.664 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:44:28.811 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:44:28.819 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:44:58.991 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:44:58.999 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:45:29.198 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:45:29.207 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:45:59.386 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:45:59.394 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:46:29.591 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:46:29.600 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:46:59.808 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:46:59.816 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:47:30.022 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:47:30.030 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:48:00.238 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:48:00.246 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:48:30.376 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:48:30.385 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:49:00.589 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:49:00.598 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:49:30.813 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:49:30.821 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:50:00.961 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:50:00.969 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:50:31.183 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:50:31.191 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:51:01.333 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:51:01.342 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:51:31.547 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:51:31.556 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:52:01.777 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:52:01.785 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:52:31.975 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:52:31.983 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:53:02.114 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:53:02.122 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:53:32.294 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:53:32.302 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:54:02.437 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:54:02.446 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:54:32.647 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:54:32.656 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:55:02.842 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:55:02.850 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:55:33.041 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:55:33.050 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:56:03.233 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:56:03.241 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:56:33.387 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:56:33.395 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:57:03.573 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:57:03.582 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:57:33.786 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:57:33.795 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:58:04.012 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:58:04.022 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:58:34.234 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:58:34.243 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:59:04.482 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:59:04.491 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:59:34.676 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:59:34.685 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:00:04.880 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:00:04.889 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:00:35.102 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:00:35.111 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:01:05.324 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:01:05.332 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:01:35.535 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:01:35.544 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:02:05.679 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:02:05.688 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:02:35.818 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:02:35.826 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:03:06.045 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:03:06.054 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:03:36.251 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:03:36.260 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:04:06.450 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:04:06.459 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:04:36.642 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:04:36.650 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:05:06.847 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:05:06.856 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:05:37.036 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:05:37.045 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:06:07.268 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:06:07.276 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:06:37.473 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:06:37.482 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:07:07.603 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:07:07.611 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:07:37.811 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:07:37.819 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:08:08.143 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:08:08.151 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:08:38.353 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:08:38.362 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:09:08.543 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:09:08.551 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:09:38.756 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:09:38.765 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:10:08.978 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:10:08.986 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:10:39.194 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:10:39.202 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:11:09.393 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:11:09.401 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:11:39.598 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:11:39.606 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:12:09.737 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:12:09.745 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:12:39.869 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:12:39.878 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:13:10.076 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:13:10.084 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:13:40.274 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:13:40.283 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:14:10.473 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:14:10.482 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:14:40.703 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:14:40.712 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:15:10.905 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:15:10.913 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:15:41.126 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:15:41.135 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:16:11.255 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:16:11.264 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:16:41.479 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:16:41.488 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:17:11.695 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:17:11.703 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:17:41.843 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:17:41.851 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:18:11.982 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:18:11.990 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:18:42.189 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:18:42.198 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:19:12.403 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:19:12.412 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:19:42.611 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:19:42.620 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:20:12.817 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:20:12.825 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:20:42.949 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:20:42.957 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:21:13.136 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:21:13.144 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:21:43.359 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:21:43.368 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:22:13.564 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:22:13.572 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:22:43.781 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:22:43.790 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:23:13.958 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:23:13.966 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:23:44.175 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:23:44.183 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:24:14.374 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:24:14.382 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:24:44.583 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:24:44.591 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:25:14.731 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:25:14.740 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:25:44.902 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:25:44.910 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:26:15.099 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:26:15.107 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:26:45.296 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:26:45.304 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:27:15.509 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:27:15.518 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:27:45.721 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:27:45.730 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:28:15.935 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:28:15.944 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:28:46.139 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:28:46.148 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:29:16.280 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:29:16.288 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:29:46.511 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:29:46.520 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:30:16.744 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:30:16.752 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:30:46.976 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:30:46.985 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:31:17.134 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:31:17.144 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:31:47.362 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:31:47.371 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:32:17.574 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:32:17.582 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:32:47.781 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:32:47.790 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:33:18.131 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:33:18.139 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:33:48.272 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:33:48.281 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:34:18.484 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:34:18.492 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:34:48.711 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:34:48.720 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:35:18.903 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:35:18.912 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:35:49.126 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:35:49.134 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:36:19.403 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:36:19.412 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:36:49.600 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:36:49.608 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:37:19.815 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:37:19.823 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:37:49.947 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:37:49.956 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:38:20.178 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:38:20.187 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:38:50.339 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:38:50.348 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:39:20.549 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:39:20.557 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:39:50.767 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:39:50.776 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:40:20.996 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:40:21.004 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:40:51.205 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:40:51.215 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:41:21.350 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:41:21.360 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:41:51.501 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:41:51.510 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:42:21.704 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:42:21.713 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:42:51.919 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:42:51.927 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:43:22.131 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:43:22.140 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:43:52.505 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:43:52.513 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:44:22.636 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:44:22.644 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:44:52.875 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:44:52.884 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:45:23.114 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:45:23.122 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:45:53.310 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:45:53.319 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:46:23.508 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:46:23.517 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:46:53.740 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:46:53.748 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:47:23.963 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:47:23.972 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:47:54.187 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:47:54.196 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:48:24.409 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:48:24.418 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:48:54.584 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:48:54.592 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:49:24.780 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:49:24.789 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:49:54.922 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:49:54.931 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:50:25.107 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:50:25.116 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:50:55.330 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:50:55.338 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:51:25.536 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:51:25.545 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:51:55.679 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:51:55.688 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:52:25.894 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:52:25.903 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:52:56.082 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:52:56.091 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:53:26.298 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:53:26.306 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:53:56.520 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:53:56.528 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:54:26.738 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:54:26.746 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:54:56.931 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:54:56.939 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:55:27.136 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:55:27.144 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:55:57.402 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:55:57.411 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:56:27.543 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:56:27.551 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:56:57.777 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:56:57.786 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:57:27.981 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:57:27.989 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:57:58.181 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:57:58.189 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:58:28.404 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:58:28.413 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:58:58.662 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:58:58.670 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:59:28.814 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:59:28.823 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:59:59.036 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:59:59.044 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:00:29.267 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:00:29.276 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:00:59.489 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:00:59.497 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:01:29.695 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:01:29.703 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:01:59.900 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:01:59.909 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:02:30.089 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:02:30.097 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:03:00.311 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:03:00.320 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:03:30.536 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:03:30.545 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:04:00.759 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:04:00.768 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:04:30.900 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:04:30.909 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:05:01.116 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:05:01.125 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:05:31.328 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:05:31.337 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:06:01.524 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:06:01.533 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:06:31.746 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:06:31.756 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:07:01.885 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:07:01.895 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:07:32.034 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:07:32.043 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:08:02.176 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:08:02.188 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:08:32.416 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:08:32.425 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:09:02.640 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:09:02.649 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:09:32.852 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:09:32.861 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:10:03.057 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:10:03.066 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:10:33.213 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:10:33.221 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:11:03.427 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:11:03.436 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:11:33.642 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:11:33.651 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:12:03.866 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:12:03.874 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:12:34.082 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:12:34.091 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:13:04.314 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:13:04.322 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:13:34.637 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:13:34.646 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:14:04.778 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:14:04.786 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:14:35.011 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:14:35.019 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:15:05.242 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:15:05.250 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:15:35.441 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:15:35.450 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:16:05.690 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:16:05.698 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:16:35.897 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:16:35.905 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:17:06.119 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:17:06.127 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:17:36.326 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:17:36.334 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:18:06.548 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:18:06.557 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:18:36.754 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:18:36.763 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:19:06.977 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:19:06.986 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:19:37.210 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:19:37.218 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:20:07.449 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:20:07.458 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:20:37.663 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:20:37.671 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:21:07.881 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:21:07.889 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:21:38.097 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:21:38.106 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:22:08.312 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:22:08.321 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:22:38.560 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:22:38.568 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:23:08.702 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:23:08.710 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:23:38.916 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:23:38.925 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:24:09.126 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:24:09.135 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:24:39.331 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:24:39.340 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:25:09.547 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:25:09.555 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:25:39.785 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:25:39.794 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:26:09.928 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:26:09.936 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:26:40.152 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:26:40.161 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:27:10.359 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:27:10.369 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:27:40.574 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:27:40.583 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:28:10.787 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:28:10.796 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:28:40.997 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:28:41.006 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:29:11.204 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:29:11.213 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:29:41.428 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:29:41.436 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:30:11.637 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:30:11.645 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:30:41.844 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:30:41.853 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:31:12.059 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:31:12.068 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:31:42.281 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:31:42.290 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:32:12.508 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:32:12.517 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:32:42.703 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:32:42.711 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:33:12.854 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:33:12.862 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:33:43.067 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:33:43.076 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:34:13.259 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:34:13.267 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:34:43.458 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:34:43.466 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:35:13.671 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:35:13.680 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:35:43.897 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:35:43.905 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:36:14.046 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:36:14.054 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:36:44.261 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:36:44.289 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:37:14.418 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:37:14.427 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:37:44.636 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:37:44.645 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:38:14.767 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:38:14.775 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:38:45.115 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:38:45.124 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:39:15.263 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:39:15.272 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:39:45.471 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:39:45.480 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:40:15.678 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:40:15.690 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:40:45.915 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:40:45.923 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:41:16.117 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:41:16.127 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:41:46.333 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:41:46.343 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:42:16.529 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:42:16.537 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:42:46.727 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:42:46.736 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:43:17.090 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:43:17.099 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:43:47.326 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:43:47.335 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:44:17.541 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:44:17.549 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:44:47.765 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:44:47.773 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:45:17.989 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:45:17.998 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:45:48.232 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:45:48.240 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:46:18.368 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:46:18.376 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:46:48.522 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:46:48.531 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:47:18.674 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:47:18.683 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:47:48.891 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:47:48.899 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:48:19.028 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:48:19.036 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:48:49.251 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:48:49.260 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:49:19.439 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:49:19.448 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:49:49.655 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:49:49.666 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:50:19.801 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:50:19.809 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:50:49.973 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:50:49.982 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:51:20.120 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:51:20.129 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:51:50.352 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:51:50.361 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:52:20.533 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:52:20.542 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:52:50.757 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:52:50.765 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:53:20.947 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:53:20.955 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:53:51.161 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:53:51.170 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:54:21.374 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:54:21.384 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:54:51.580 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:54:51.588 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:55:21.785 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:55:21.794 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:55:52.008 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:55:52.017 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:56:22.221 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:56:22.229 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:56:52.438 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:56:52.446 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:57:22.635 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:57:22.644 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:57:52.818 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:57:52.828 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:58:23.042 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:58:23.051 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:58:53.257 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:58:53.265 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:59:23.472 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:59:23.480 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:59:53.705 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:59:53.713 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:00:23.854 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:00:23.863 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:00:54.066 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:00:54.075 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:01:24.255 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:01:24.263 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:01:54.460 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:01:54.468 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:02:24.665 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:02:24.673 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:02:55.049 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:02:55.085 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:03:25.223 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:03:25.232 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:03:55.363 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:03:55.371 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:04:25.578 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:04:25.587 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:04:55.766 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:04:55.775 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:05:25.998 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:05:26.007 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:05:56.319 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:05:56.349 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:06:26.560 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:06:26.569 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:06:56.783 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:06:56.791 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:07:27.023 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:07:27.033 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:07:57.224 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:07:57.232 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:08:27.455 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:08:27.464 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:08:57.703 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:08:57.711 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:09:27.855 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:09:27.864 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:09:58.053 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:09:58.061 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:10:28.267 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:10:28.275 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:10:58.466 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:10:58.474 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:11:28.729 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:11:28.738 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:11:58.867 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:11:58.875 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:12:29.121 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:12:29.129 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:12:59.252 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:12:59.263 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:13:29.494 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:13:29.502 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:13:59.735 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:13:59.744 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:14:29.874 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:14:29.882 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:15:00.098 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:15:00.107 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:15:30.288 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:15:30.296 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:16:00.507 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:16:00.515 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:16:30.643 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:16:30.652 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:17:00.851 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:17:00.859 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:17:31.089 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:17:31.097 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:18:01.298 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:18:01.307 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:18:31.531 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:18:31.538 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:19:01.746 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:19:01.755 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:19:31.885 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:19:31.894 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:20:02.082 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:20:02.090 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:20:32.296 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:20:32.305 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:21:02.710 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:21:02.719 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:21:32.908 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:21:32.917 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:22:03.114 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:22:03.123 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:22:33.329 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:22:33.338 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:23:03.541 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:23:03.550 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:23:33.730 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:23:33.739 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:24:03.932 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:24:03.941 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:24:34.138 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:24:34.149 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:25:04.352 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:25:04.361 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:25:34.580 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:25:34.588 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:26:04.806 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:26:04.815 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:26:34.957 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:26:34.966 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:27:05.154 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:27:05.162 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:27:35.347 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:27:35.356 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:28:05.572 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:28:05.580 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:28:35.794 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:28:35.802 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:29:06.010 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:29:06.018 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:29:36.214 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:29:36.222 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:30:06.411 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:30:06.419 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:30:36.620 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:30:36.628 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:31:06.851 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:31:06.859 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:31:36.992 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:31:37.000 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:32:07.221 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:32:07.230 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:32:37.419 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:32:37.428 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:33:07.642 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:33:07.651 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:33:37.850 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:33:37.858 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:34:08.089 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:34:08.099 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:34:38.295 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:34:38.303 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:35:08.517 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:35:08.526 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:35:38.724 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:35:38.733 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:36:08.938 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:36:08.948 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:36:39.151 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:36:39.160 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:37:09.376 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:37:09.385 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:37:39.565 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:37:39.575 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:38:09.772 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:38:09.781 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:38:39.990 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:38:39.998 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:39:10.133 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:39:10.142 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:39:40.397 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:39:40.406 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:40:10.579 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:40:10.587 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:40:40.810 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:40:40.818 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:41:11.022 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:41:11.031 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:41:41.222 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:41:41.231 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:42:11.428 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:42:11.437 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:42:41.599 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:42:41.607 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:43:11.791 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:43:11.800 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:43:41.953 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:43:41.962 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:44:12.172 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:44:12.180 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:44:42.386 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:44:42.397 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:45:12.627 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:45:12.636 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:45:42.785 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:45:42.794 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:46:13.116 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:46:13.124 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:46:43.330 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:46:43.338 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:47:13.554 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:47:13.563 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:47:43.767 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:47:43.777 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:48:13.970 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:48:13.979 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:48:44.190 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:48:44.199 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:49:14.397 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:49:14.407 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:49:44.603 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:49:44.612 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:50:14.742 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:50:14.751 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:50:44.960 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:50:44.970 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:51:15.164 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:51:15.173 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:51:45.378 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:51:45.387 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:52:15.551 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:52:15.560 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:52:45.781 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:52:45.790 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:53:16.014 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:53:16.022 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:53:46.225 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:53:46.233 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:54:16.416 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:54:16.426 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:54:46.659 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:54:46.668 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:55:16.810 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:55:16.818 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:55:47.008 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:55:47.017 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:56:17.196 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:56:17.205 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:56:47.417 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:56:47.426 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:57:17.641 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:57:17.649 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:57:47.839 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:57:47.848 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:58:18.052 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:58:18.061 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:58:48.266 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:58:48.276 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:59:18.480 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:59:18.489 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:59:48.687 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:59:48.695 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:00:18.873 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:00:18.882 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:00:49.120 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:00:49.128 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:01:19.323 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:01:19.332 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:01:49.538 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:01:49.547 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:02:19.754 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:02:19.762 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:02:49.903 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:02:49.916 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:03:20.125 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:03:20.133 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:03:50.340 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:03:50.349 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:04:20.565 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:04:20.574 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:04:50.753 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:04:50.762 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:05:20.950 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:05:20.959 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:05:51.168 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:05:51.176 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:06:21.391 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:06:21.400 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:06:51.531 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:06:51.540 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:07:21.722 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:07:21.730 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:07:51.911 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:07:51.919 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:08:22.099 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:08:22.108 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:08:52.315 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:08:52.324 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:09:22.521 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:09:22.530 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:09:52.874 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:09:52.882 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:10:23.019 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:10:23.028 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:10:53.210 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:10:53.218 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:11:23.416 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:11:23.424 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:11:53.639 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:11:53.647 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:12:23.869 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:12:23.877 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:12:54.082 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:12:54.090 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:13:24.295 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:13:24.303 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:13:54.575 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:13:54.584 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:14:24.794 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:14:24.804 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:14:54.995 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:14:55.003 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:15:25.134 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:15:25.142 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:15:55.351 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:15:55.359 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:16:25.564 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:16:25.573 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:16:55.771 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:16:55.780 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:17:26.057 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:17:26.065 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:17:56.194 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:17:56.207 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:18:26.416 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:18:26.425 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:18:56.641 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:18:56.651 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:19:26.847 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:19:26.859 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:19:56.997 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:19:57.006 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:20:27.127 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:20:27.136 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:20:57.322 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:20:57.331 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:21:27.502 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:21:27.510 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:21:57.748 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:21:57.757 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:22:27.964 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:22:27.974 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:22:58.177 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:22:58.185 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:23:28.411 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:23:28.420 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:23:58.649 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:23:58.658 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:24:28.830 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:24:28.839 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:24:59.054 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:24:59.066 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:25:29.220 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:25:29.228 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:25:59.436 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:25:59.445 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:26:29.651 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:26:29.660 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:26:59.847 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:26:59.855 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:27:30.057 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:27:30.066 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:28:00.275 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:28:00.284 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:28:30.498 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:28:30.507 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:29:00.696 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:29:00.705 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:29:30.919 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:29:30.928 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:30:01.138 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:30:01.147 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:30:31.350 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:30:31.358 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:31:01.497 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:31:01.506 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:31:31.721 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:31:31.730 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:32:01.937 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:32:01.948 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:32:32.162 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:32:32.171 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:33:02.400 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:33:02.410 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:33:32.633 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:33:32.641 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:34:02.857 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:34:02.869 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:34:33.056 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:34:33.064 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:35:03.269 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:35:03.277 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:35:33.493 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:35:33.502 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:36:03.717 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:36:03.726 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:36:33.950 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:36:33.959 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:37:04.166 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:37:04.175 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:37:34.321 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:37:34.329 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:38:04.587 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:38:04.595 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:38:34.808 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:38:34.817 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:39:05.022 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:39:05.031 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:39:35.245 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:39:35.254 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:40:05.466 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:40:05.475 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:40:35.699 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:40:35.707 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:41:05.912 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:41:05.922 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:41:36.110 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:41:36.119 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:42:06.341 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:42:06.350 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:42:36.542 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:42:36.551 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:43:06.695 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:43:06.704 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:43:36.927 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:43:36.936 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:44:07.123 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:44:07.131 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:44:37.334 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:44:37.352 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:45:07.502 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:45:07.511 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:45:37.711 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:45:37.720 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:46:07.933 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:46:07.942 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:46:38.142 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:46:38.151 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:47:08.366 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:47:08.375 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:47:38.587 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:47:38.595 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:48:08.805 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:48:08.813 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:48:39.006 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:48:39.016 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:49:09.422 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:49:09.433 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:49:39.568 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:49:39.576 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:50:09.707 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:50:09.716 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:50:39.931 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:50:39.939 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:51:10.127 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:51:10.136 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:51:40.324 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:51:40.333 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:52:10.739 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:52:10.749 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:52:40.877 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:52:40.886 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:53:11.093 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:53:11.103 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:53:41.318 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:53:41.328 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:54:11.540 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:54:11.548 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:54:41.744 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:54:41.753 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:55:11.909 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:55:11.919 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:55:42.057 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:55:42.066 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:56:12.274 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:56:12.285 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:56:42.495 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:56:42.503 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:57:12.691 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:57:12.700 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:57:43.065 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:57:43.074 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:58:13.215 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:58:13.227 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:58:43.429 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:58:43.443 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:59:13.578 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:59:13.589 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:59:43.798 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:59:43.807 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:00:14.018 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:00:14.027 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:00:44.240 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:00:44.248 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:01:14.436 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:01:14.444 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:01:44.650 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:01:44.658 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:02:14.864 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:02:14.873 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:02:45.020 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:02:45.030 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:03:15.216 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:03:15.226 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:03:45.429 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:03:45.438 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:04:15.617 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:04:15.626 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:04:45.806 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:04:45.814 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:05:15.971 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:05:15.981 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:05:46.187 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:05:46.196 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:06:16.387 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:06:16.396 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:06:46.608 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:06:46.616 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:07:16.781 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:07:16.789 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:07:47.145 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:07:47.154 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:08:17.343 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:08:17.352 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:08:47.558 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:08:47.566 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:09:17.763 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:09:17.772 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:09:47.962 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:09:47.971 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:10:18.184 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:10:18.192 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:10:48.431 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:10:48.439 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:11:18.654 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:11:18.662 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:11:48.933 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:11:48.941 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:12:19.074 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:12:19.082 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:12:49.260 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:12:49.269 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:13:19.463 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:13:19.473 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:13:49.695 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:13:49.704 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:14:19.893 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:14:19.902 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:14:50.114 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:14:50.124 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:15:20.364 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:15:20.373 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:15:50.500 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:15:50.508 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:16:20.703 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:16:20.712 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:16:50.918 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:16:50.926 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:17:21.130 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:17:21.139 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:17:51.357 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:17:51.367 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:18:21.573 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:18:21.582 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:18:51.777 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:18:51.786 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:19:21.916 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:19:21.924 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:19:52.112 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:19:52.121 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:20:22.468 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:20:22.480 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:20:52.617 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:20:52.626 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:21:22.839 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:21:22.848 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:21:53.042 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:21:53.051 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:22:23.243 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:22:23.252 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:22:53.466 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:22:53.476 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:23:23.647 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:23:23.656 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:23:53.898 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:23:53.907 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:24:24.061 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:24:24.070 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:24:54.289 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:24:54.298 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:25:24.486 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:25:24.495 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:25:54.734 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:25:54.744 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:26:24.902 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:26:24.911 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:26:55.125 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:26:55.133 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:27:25.318 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:27:25.327 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:27:55.524 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:27:55.532 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:28:25.738 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:28:25.747 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:28:55.906 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:28:55.915 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:29:26.076 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:29:26.085 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:29:56.299 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:29:56.307 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:30:26.497 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:30:26.505 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:30:56.638 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:30:56.647 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:31:26.819 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:31:26.828 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:31:57.024 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:31:57.032 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:32:27.230 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:32:27.238 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:32:57.460 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:32:57.468 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:33:27.687 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:33:27.696 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:33:57.837 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:33:57.846 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:34:28.058 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:34:28.066 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:34:58.215 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:34:58.223 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:35:28.429 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:35:28.438 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:35:58.718 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:35:58.727 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:36:28.932 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:36:28.940 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:36:59.180 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:36:59.188 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:37:29.389 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:37:29.398 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:37:59.621 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:37:59.630 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:38:29.829 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:38:29.838 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:39:00.020 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:39:00.028 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:39:30.225 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:39:30.234 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:40:00.368 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:40:00.377 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:40:30.533 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:40:30.542 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:41:00.748 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:41:00.757 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:41:30.897 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:41:30.906 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:42:01.121 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:42:01.130 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:42:31.346 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:42:31.355 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:43:01.577 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:43:01.586 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:43:31.791 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:43:31.799 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:44:01.962 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:44:01.974 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:44:32.152 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:44:32.161 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:45:02.390 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:45:02.398 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:45:32.605 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:45:32.614 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:46:02.820 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:46:02.829 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:46:33.170 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:46:33.179 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:47:03.309 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:47:03.318 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:47:33.515 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:47:33.524 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:48:03.744 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:48:03.752 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:48:33.949 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:48:33.958 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:49:04.170 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:49:04.179 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:49:34.436 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:49:34.445 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:50:04.730 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:50:04.738 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:50:34.944 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:50:34.953 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:51:05.174 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:51:05.183 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:51:35.373 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:51:35.382 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:52:05.567 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:52:05.576 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:52:35.770 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:52:35.779 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:53:05.971 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:53:05.980 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:53:36.192 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:53:36.201 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:54:06.408 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:54:06.416 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:54:36.614 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:54:36.623 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:55:06.847 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:55:06.856 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:55:37.041 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:55:37.049 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:56:07.245 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:56:07.254 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:56:37.460 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:56:37.468 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:57:07.665 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:57:07.676 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:57:37.979 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:57:37.988 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:58:08.185 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:58:08.194 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:58:38.389 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:58:38.398 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:59:08.585 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:59:08.593 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:59:38.798 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:59:38.807 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:00:09.020 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:00:09.029 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:00:39.244 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:00:39.255 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:01:09.453 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:01:09.462 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:01:39.598 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:01:39.607 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:02:09.820 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:02:09.828 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:02:40.015 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:02:40.024 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:03:10.205 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:03:10.213 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:03:40.421 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:03:40.429 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:04:10.569 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:04:10.577 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:04:40.788 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:04:40.797 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:05:10.993 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:05:11.001 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:05:41.141 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:05:41.150 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:06:11.354 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:06:11.363 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:06:41.579 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:06:41.588 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:07:11.777 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:07:11.785 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:07:41.932 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:07:41.941 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:08:12.069 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:08:12.077 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:08:42.271 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:08:42.280 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:09:12.494 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:09:12.503 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:09:42.718 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:09:42.726 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:10:12.858 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:10:12.867 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:10:43.087 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:10:43.096 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:11:13.292 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:11:13.301 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:11:43.498 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:11:43.506 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:12:13.703 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:12:13.712 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:12:43.842 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:12:43.851 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:13:14.058 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:13:14.067 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:13:44.366 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:13:44.374 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:14:14.505 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:14:14.515 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:14:44.718 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:14:44.728 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:15:14.861 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:15:14.870 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:15:45.068 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:15:45.076 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:16:15.291 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:16:15.300 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:16:45.512 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:16:45.520 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:17:15.742 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:17:15.752 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:17:45.971 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:17:45.980 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:18:16.202 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:18:16.211 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:18:46.415 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:18:46.423 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:19:16.638 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:19:16.647 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:19:46.843 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:19:46.852 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:20:16.978 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:20:16.987 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:20:47.197 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:20:47.206 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:21:17.407 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:21:17.416 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:21:47.744 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:21:47.755 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:22:17.958 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:22:17.967 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:22:48.173 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:22:48.182 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:23:18.445 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:23:18.454 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:23:48.644 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:23:48.653 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:24:18.858 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:24:18.866 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:24:49.039 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:24:49.048 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:25:19.237 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:25:19.245 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:25:49.432 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:25:49.441 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:26:19.613 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:26:19.622 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:26:49.758 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:26:49.766 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:27:19.956 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:27:19.965 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:27:50.098 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:27:50.106 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:28:20.305 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:28:20.315 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:28:50.507 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:28:50.516 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:29:20.706 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:29:20.714 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:29:50.919 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:29:50.927 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:30:21.149 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:30:21.158 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:30:51.299 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:30:51.308 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:31:21.438 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:31:21.446 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:31:51.578 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:31:51.586 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:32:21.726 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:32:21.735 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:32:51.891 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:32:51.902 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:33:22.097 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:33:22.106 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:33:52.263 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:33:52.271 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:34:22.456 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:34:22.465 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:34:52.696 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:34:52.705 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:35:22.901 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:35:22.910 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:35:53.041 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:35:53.049 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:36:23.242 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:36:23.251 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:36:53.456 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:36:53.465 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:37:23.663 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:37:23.672 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:37:53.903 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:37:53.912 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:38:24.120 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:38:24.128 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:38:54.343 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:38:54.353 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:39:24.546 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:39:24.555 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:39:54.751 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:39:54.759 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:40:24.973 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:40:24.982 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:40:55.179 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:40:55.187 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:41:25.393 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:41:25.402 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:41:55.533 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:41:55.543 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:42:25.740 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:42:25.748 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:42:55.886 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:42:55.895 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:43:26.102 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:43:26.110 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:43:56.248 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:43:56.257 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:44:26.446 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:44:26.454 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:44:56.593 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:44:56.601 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:45:26.800 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:45:26.809 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:45:56.974 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:45:56.983 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:46:27.114 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:46:27.122 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:46:57.312 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:46:57.320 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:47:27.526 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:47:27.535 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:47:57.746 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:47:57.755 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:48:27.943 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:48:27.952 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:48:58.153 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:48:58.161 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:49:28.368 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:49:28.376 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:49:58.573 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:49:58.581 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:50:28.787 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:50:28.795 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:50:59.000 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:50:59.009 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:51:29.211 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:51:29.220 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:51:59.417 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:51:59.426 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:52:29.626 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:52:29.635 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:52:59.773 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:52:59.782 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:53:30.006 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:53:30.014 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:54:00.212 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:54:00.221 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:54:30.405 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:54:30.413 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:55:00.609 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:55:00.618 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:55:30.748 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:55:30.756 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:56:00.973 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:56:00.981 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:56:31.179 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:56:31.188 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:57:01.412 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:57:01.420 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:57:31.552 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:57:31.560 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:58:01.783 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:58:01.792 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:58:31.933 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:58:31.941 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:59:02.080 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:59:02.088 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:59:32.260 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:59:32.269 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:00:02.448 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:00:02.456 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:00:32.678 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:00:32.687 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:01:02.868 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:01:02.877 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:01:33.179 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:01:33.188 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:02:03.326 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:02:03.334 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:02:33.581 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:02:33.590 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:03:03.785 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:03:03.794 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:03:33.941 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:03:33.950 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:04:04.154 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:04:04.162 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:04:34.376 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:04:34.385 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:05:04.600 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:05:04.609 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:05:34.819 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:05:34.827 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:06:04.956 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:06:04.965 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:06:35.180 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:06:35.189 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:07:05.409 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:07:05.418 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:07:35.621 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:07:35.698 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:08:05.901 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:08:05.910 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:08:36.037 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:08:36.046 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:09:06.244 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:09:06.252 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:09:36.432 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:09:36.440 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:10:06.679 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:10:06.688 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:10:36.898 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:10:36.908 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:11:07.046 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:11:07.055 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:11:37.262 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:11:37.270 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:12:07.526 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:12:07.535 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:12:37.747 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:12:37.755 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:13:07.967 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:13:07.976 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:13:38.169 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:13:38.178 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:14:08.385 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:14:08.394 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:14:38.615 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:14:38.624 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:15:08.837 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:15:08.846 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:15:39.043 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:15:39.051 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:16:09.205 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:16:09.214 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:16:39.348 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:16:39.358 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:17:09.561 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:17:09.570 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:17:39.750 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:17:39.759 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:18:09.972 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:18:09.981 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:18:40.162 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:18:40.171 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:19:10.293 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:19:10.302 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:19:40.492 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:19:40.503 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:20:10.706 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:20:10.715 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:20:40.904 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:20:40.912 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:21:11.051 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:21:11.060 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:21:41.272 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:21:41.281 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:22:11.486 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:22:11.495 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:22:41.699 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:22:41.708 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:23:11.913 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:23:11.922 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:23:42.061 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:23:42.071 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:24:12.260 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:24:12.269 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:24:42.401 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:24:42.409 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:25:12.615 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:25:12.623 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:25:42.831 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:25:42.840 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:26:13.050 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:26:13.059 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:26:43.254 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:26:43.263 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:27:13.393 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:27:13.401 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:27:43.544 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:27:43.553 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:28:13.751 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:28:13.760 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:28:43.959 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:28:43.967 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:29:14.183 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:29:14.193 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:29:44.399 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:29:44.407 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:30:14.538 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:30:14.547 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:30:44.790 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:30:44.801 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:31:15.015 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:31:15.024 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:31:45.237 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:31:45.248 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:32:15.444 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:32:15.453 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:32:45.619 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:32:45.627 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:33:15.809 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:33:15.818 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:33:45.974 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:33:45.983 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:34:16.165 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:34:16.174 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:34:46.367 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:34:46.376 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:35:16.589 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:35:16.598 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:35:46.879 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:35:46.887 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:36:17.018 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:36:17.027 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:36:47.232 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:36:47.241 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:37:17.431 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:37:17.440 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:37:47.664 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:37:47.673 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:38:17.854 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:38:17.863 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:38:48.086 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:38:48.094 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:39:18.291 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:39:18.300 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:39:48.439 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:39:48.447 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:40:18.661 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:40:18.670 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:40:48.810 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:40:48.818 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:41:19.016 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:41:19.024 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:41:49.239 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:41:49.248 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:42:19.436 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:42:19.445 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:42:49.637 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:42:49.645 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:43:19.855 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:43:19.864 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:43:50.008 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:43:50.016 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:44:20.153 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:44:20.162 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:44:50.507 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:44:50.515 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:45:20.696 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:45:20.705 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:45:50.895 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:45:50.904 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:46:21.112 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:46:21.121 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:46:51.319 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:46:51.328 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:47:21.542 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:47:21.552 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:47:51.691 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:47:51.700 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:48:21.925 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:48:21.935 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:48:52.072 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:48:52.081 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:49:22.298 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:49:22.306 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:49:52.505 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:49:52.514 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:50:22.803 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:50:22.812 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:50:52.950 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:50:52.959 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:51:23.093 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:51:23.101 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:51:53.307 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:51:53.315 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:52:23.507 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:52:23.515 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:52:53.719 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:52:53.728 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:53:23.924 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:53:23.932 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:53:54.142 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:53:54.152 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:54:24.333 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:54:24.342 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:54:54.540 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:54:54.549 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:55:24.747 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:55:24.755 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:55:54.963 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:55:54.971 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:56:25.142 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:56:25.151 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:56:55.282 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:56:55.290 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:57:25.430 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:57:25.439 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:57:55.629 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:57:55.637 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:58:25.845 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:58:25.854 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:58:56.067 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:58:56.076 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:59:26.240 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:59:26.248 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:59:56.454 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:59:56.464 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:00:26.669 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:00:26.678 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:00:56.809 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:00:56.819 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:01:27.069 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:01:27.078 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:01:57.292 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:01:57.300 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:02:27.439 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:02:27.448 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:02:57.663 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:02:57.672 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:03:27.894 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:03:27.903 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:03:58.101 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:03:58.111 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:04:28.307 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:04:28.315 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:04:58.511 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:04:58.519 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:05:28.660 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:05:28.668 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:05:58.884 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:05:58.893 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:06:29.040 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:06:29.049 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:06:59.265 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:06:59.273 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:07:29.456 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:07:29.465 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:07:59.697 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:07:59.707 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:08:29.902 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:08:29.910 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:09:00.112 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:09:00.120 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:09:30.319 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:09:30.328 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:10:00.523 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:10:00.535 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:10:30.737 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:10:30.745 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:11:00.891 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:11:00.900 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:11:31.107 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:11:31.116 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:12:01.240 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:12:01.249 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:12:31.473 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:12:31.482 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:13:01.616 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:13:01.624 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:13:31.846 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:13:31.855 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:14:01.986 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:14:01.994 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:14:32.209 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:14:32.218 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:15:02.451 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:15:02.459 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:15:32.648 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:15:32.657 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:16:02.863 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:16:02.872 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:16:33.079 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:16:33.088 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:17:03.286 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:17:03.294 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:17:33.434 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:17:33.442 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:18:03.633 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:18:03.641 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:18:33.858 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:18:33.867 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:19:04.074 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:19:04.083 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:19:34.276 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:19:34.286 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:20:04.414 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:20:04.423 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:20:34.652 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:20:34.661 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:21:04.876 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:21:04.885 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:21:35.116 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:21:35.125 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:22:05.323 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:22:05.331 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:22:35.461 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:22:35.469 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:23:05.669 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:23:05.677 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:23:35.901 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:23:35.910 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:24:06.132 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:24:06.141 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:24:36.339 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:24:36.348 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:25:06.486 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:25:06.494 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:25:36.700 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:25:36.709 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:26:06.842 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:26:06.851 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:26:37.054 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:26:37.062 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:27:07.193 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:27:07.202 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:27:37.336 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:27:37.345 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:28:07.475 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:28:07.484 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:28:37.615 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:28:37.623 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:29:07.762 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:29:07.771 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:29:37.952 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:29:37.961 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:30:08.176 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:30:08.185 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:30:38.384 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:30:38.392 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:31:08.549 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:31:08.557 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:31:38.696 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:31:38.705 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:32:08.904 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:32:08.913 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:32:39.111 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:32:39.120 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:33:09.322 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:33:09.331 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:33:39.522 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:33:39.531 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:34:09.738 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:34:09.747 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:34:39.935 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:34:39.943 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:35:10.166 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:35:10.175 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:35:40.371 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:35:40.380 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:36:10.610 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:36:10.619 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:36:40.825 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:36:40.833 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:37:11.030 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:37:11.039 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:37:41.236 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:37:41.244 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:38:11.450 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:38:11.459 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:38:41.615 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:38:41.623 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:39:11.829 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:39:11.838 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:39:41.968 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:39:41.977 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:40:12.115 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:40:12.124 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:40:42.272 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:40:42.281 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:41:12.487 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:41:12.496 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:41:42.678 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:41:42.687 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:42:12.827 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:42:12.836 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:42:43.051 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:42:43.060 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:43:13.283 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:43:13.291 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:43:43.517 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:43:43.526 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:44:13.744 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:44:13.753 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:44:43.966 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:44:43.975 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:45:14.170 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:45:14.178 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:45:44.350 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:45:44.358 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:46:14.546 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:46:14.554 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:46:44.713 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:46:44.722 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:47:14.928 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:47:14.936 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:47:45.144 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:47:45.152 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:48:15.351 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:48:15.360 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:48:45.491 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:48:45.500 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:49:15.709 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:49:15.718 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:49:45.914 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:49:45.923 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:50:16.129 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:50:16.138 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:50:46.352 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:50:46.361 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:51:16.619 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:51:16.627 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:51:46.842 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:51:46.851 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:52:17.057 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:52:17.066 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:52:47.275 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:52:47.283 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:53:17.497 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:53:17.506 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:53:47.696 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:53:47.704 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:54:17.826 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:54:17.834 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:54:48.050 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:54:48.058 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:55:18.282 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:55:18.291 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:55:48.498 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:55:48.507 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:56:18.717 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:56:18.725 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:56:48.873 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:56:48.882 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:57:19.115 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:57:19.124 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:57:49.353 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:57:49.362 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:58:19.568 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:58:19.578 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:58:49.773 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:58:49.783 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:59:19.913 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:59:19.922 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:59:50.144 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:59:50.153 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:00:20.376 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:00:20.384 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:00:50.582 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:00:50.591 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:01:20.804 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:01:20.813 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:01:50.944 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:01:50.953 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:02:21.161 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:02:21.170 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:02:51.393 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:02:51.401 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:03:21.606 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:03:21.614 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:03:51.832 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:03:51.840 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:04:21.971 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:04:21.980 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:04:52.172 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:04:52.181 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:05:22.393 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:05:22.402 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:05:52.609 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:05:52.618 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:06:22.815 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:06:22.824 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:06:53.005 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:06:53.014 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:07:23.183 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:07:23.192 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:07:53.383 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:07:53.391 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:08:23.572 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:08:23.581 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:08:53.794 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:08:53.803 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:09:23.976 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:09:23.990 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:09:54.123 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:09:54.132 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:10:24.336 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:10:24.344 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:10:54.485 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:10:54.494 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:11:24.680 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:11:24.689 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:11:54.817 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:11:54.826 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:12:24.956 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:12:24.964 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:12:55.095 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:12:55.103 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:13:25.227 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:13:25.235 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:13:55.430 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:13:55.439 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:14:25.659 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:14:25.667 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:14:55.864 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:14:55.872 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:15:26.061 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:15:26.069 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:15:56.193 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:15:56.201 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:16:26.403 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:16:26.412 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:16:56.608 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:16:56.616 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:17:26.815 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:17:26.823 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:17:56.996 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:17:57.005 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:18:27.213 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:18:27.221 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:18:57.445 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:18:57.454 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:19:27.651 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:19:27.660 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:19:57.791 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:19:57.800 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:20:27.932 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:20:27.940 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:20:58.155 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:20:58.164 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:21:28.378 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:21:28.387 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:21:58.601 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:21:58.609 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:22:28.807 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:22:28.816 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:22:59.030 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:22:59.038 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:23:29.204 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:23:29.212 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:23:59.411 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:23:59.419 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:24:29.660 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:24:29.668 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:24:59.883 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:24:59.892 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:25:30.082 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:25:30.091 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:26:00.223 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:26:00.231 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:26:30.373 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:26:30.381 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:27:00.589 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:27:00.598 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:27:30.812 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:27:30.821 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:28:01.010 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:28:01.018 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:28:31.243 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:28:31.251 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:29:01.483 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:29:01.491 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:29:31.639 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:29:31.648 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:30:01.770 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:30:01.778 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:30:31.986 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:30:31.995 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:31:02.193 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:31:02.201 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:31:32.383 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:31:32.391 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:32:02.590 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:32:02.599 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:32:32.965 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:32:32.974 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:33:03.285 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:33:03.294 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:33:33.426 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:33:33.435 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:34:03.557 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:34:03.567 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:34:33.761 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:34:33.770 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:35:03.970 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:35:03.979 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:35:34.199 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:35:34.208 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:36:04.359 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:36:04.369 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:36:34.505 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:36:34.514 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:37:04.721 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:37:04.729 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:37:34.893 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:37:34.901 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:38:05.034 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:38:05.043 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:38:35.298 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:38:35.307 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:39:05.453 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:39:05.462 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:39:35.592 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:39:35.601 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:40:05.833 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:40:05.842 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:40:36.029 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:40:36.038 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:41:06.453 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:41:06.462 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:41:36.650 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:41:36.658 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:42:06.784 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:42:06.793 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:42:36.952 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:42:36.961 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:43:07.099 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:43:07.108 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:43:37.234 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:43:37.243 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:44:07.364 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:44:07.373 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:44:37.496 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:44:37.505 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:45:07.636 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:45:07.645 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:45:37.859 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:45:37.868 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:46:08.056 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:46:08.066 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:46:38.270 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:46:38.279 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:47:08.459 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:47:08.467 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:47:38.650 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:47:38.660 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:48:08.796 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:48:08.805 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:48:39.001 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:48:39.009 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:49:09.189 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:49:09.198 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:49:39.339 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:49:39.575 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:50:09.712 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:50:09.721 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:50:39.849 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:50:39.858 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:51:10.060 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:51:10.068 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:51:40.204 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:51:40.212 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:52:10.419 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:52:10.428 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:52:40.565 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:52:40.574 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:53:10.770 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:53:10.778 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:53:40.926 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:53:40.934 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:54:11.150 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:54:11.159 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:54:41.365 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:54:41.417 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:55:11.654 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:55:11.662 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:55:41.878 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:55:41.887 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:56:12.018 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:56:12.027 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:56:42.243 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:56:42.252 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:57:12.461 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:57:12.469 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:57:42.607 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:57:42.616 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:58:12.804 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:58:12.813 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:58:43.043 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:58:43.052 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:59:13.254 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:59:13.263 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:59:43.444 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:59:43.452 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:00:13.650 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:00:13.659 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:00:43.825 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:00:43.834 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:01:14.108 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:01:14.117 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:01:44.253 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:01:44.262 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:02:14.474 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:02:14.483 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:02:44.680 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:02:44.688 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:03:14.903 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:03:14.911 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:03:45.134 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:03:45.143 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:04:15.273 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:04:15.282 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:04:45.512 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:04:45.520 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:05:15.659 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:05:15.668 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:05:45.799 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:05:45.807 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:06:15.938 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:06:15.947 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:06:46.129 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:06:46.138 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:07:16.275 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:07:16.284 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:07:46.490 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:07:46.499 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:08:16.653 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:08:16.662 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:08:46.875 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:08:46.883 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:09:17.016 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:09:17.025 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:09:47.210 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:09:47.218 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:10:17.346 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:10:17.355 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:10:47.555 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:10:47.564 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:11:17.778 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:11:17.786 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:11:47.983 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:11:47.991 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:12:18.192 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:12:18.200 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:12:48.394 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:12:48.403 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:13:18.587 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:13:18.595 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:13:48.840 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:13:48.848 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:14:19.079 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:14:19.088 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:14:49.298 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:14:49.306 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:15:19.437 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:15:19.446 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:15:49.633 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:15:49.642 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:16:19.782 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:16:19.790 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:16:49.912 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:16:49.921 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:17:20.056 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:17:20.064 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:17:50.191 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:17:50.200 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:18:20.423 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:18:20.432 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:18:50.654 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:18:50.663 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:19:20.802 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:19:20.810 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:19:50.943 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:19:50.951 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:20:21.074 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:20:21.082 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:20:51.211 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:20:51.219 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:21:21.408 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:21:21.417 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:21:51.614 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:21:51.623 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:22:21.800 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:22:21.808 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:22:51.933 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:22:51.942 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:23:22.078 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:23:22.086 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:23:52.284 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:23:52.293 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:24:22.481 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:24:22.489 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:24:52.627 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:24:52.636 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:25:22.824 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:25:22.832 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:25:52.995 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:25:53.004 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:26:23.276 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:26:23.284 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:26:53.418 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:26:53.427 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:27:23.617 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:27:23.626 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:27:53.842 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:27:53.850 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:28:24.080 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:28:24.090 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:28:54.305 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:28:54.314 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:29:24.516 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:29:24.524 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:29:54.747 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:29:54.756 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:30:24.968 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:30:24.977 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:30:55.109 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:30:55.118 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:31:25.252 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:31:25.260 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:31:55.465 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:31:55.474 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:32:25.672 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:32:25.680 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:32:55.843 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:32:55.852 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:33:26.057 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:33:26.065 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:33:56.237 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:33:56.246 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:34:26.451 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:34:26.460 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:34:56.656 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:34:56.665 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:35:26.885 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:35:26.893 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:35:57.025 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:35:57.034 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:36:27.229 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:36:27.238 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:36:57.426 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:36:57.435 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:37:27.623 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:37:27.631 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:37:57.827 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:37:57.836 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:38:27.968 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:38:27.976 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:38:58.100 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:38:58.108 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:39:28.241 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:39:28.274 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:39:58.405 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:39:58.414 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:40:28.542 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:40:28.551 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:40:58.677 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:40:58.686 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:41:28.811 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:41:28.821 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:41:59.014 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:41:59.022 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:42:29.254 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:42:29.264 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:42:59.402 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:42:59.411 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:43:29.608 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:43:29.616 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:43:59.840 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:43:59.848 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:44:30.023 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:44:30.032 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:44:34.935 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2024-01-17T02:44:34.935 DEBUG:teuthology.task.console_log:Killing console logger for smithi143 2024-01-17T02:44:34.936 DEBUG:teuthology.task.console_log:Killing console logger for smithi204 2024-01-17T02:44:34.936 DEBUG:teuthology.exit:Finished running handlers